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ABSTRACT ^ * . - ' . , y 



An experiment comparigig; the effectiveness ojf three design 
strategies used nine architecture graduate students solving 
three "typical i?oom iiesign pJroblems. Open-ended (5-step) , 
traditional (f 0-step) , and systematic Cl5-step) strategies 
were developed' based .on a national survey of design method- 
ologists'. Each strategy vas applied by. three subjects 
working independently in three 8-hour test sessions t:o pro- 
vide desigr^s for a faculty o.f fice, campus snack bar,^ ^nd 
departinipntal conference room. \ 

• i 

The design process was recorded every 30 secondsi when 
a camera photographed the wbrk area on which subjects disr 
played and manipulated cards taken from an in«>rmation bank. 
The bank contained over ,800 data *^i terns on reqixirements, de- 
si^ criteria, and solution ;alternati\res ^or each problem, 
thirty- five process measures were generated including: time - 
for solution, number and type ^of cards used; nuniber and type', 
of relationships formed with the cards. Twenty-five solu- 
tion measures covered . the differences in design configuration 
and quality, including ratings of judges. • ' ' \ 

' / ' ■ ' ■ ■ ■ • ■ ' . . 

Analysis of variance indicated significant differences 

fexisted for problems and" for subjects within'^strategy groups. 
Some signifipant strategy effects were faund over all three 
problems (8 measures), on individual problems^ {16 measures)^ 
and . in stt.ategy-problem interactions (?,4 mjeasures^)''. 'fhe '5- ► 
steij. strategy required a l^rge amount of time, using large 
quantities of information minimally applied; the ,10--step 
strategy cohcentrated on a detailed problem, development with 
a low use bf conceptual information; and the 15-stet> strategy* 
operated moire efficiently overall, ^ producing nonsigniflcantly, 
but 'apparently, , better solutions . Interaction results sug- 
gested that the 5-step strategy jreacjl^ed similarlyito all 
three problems; the 10-step strategy became more efficient 
oyer the thred. sessions; and the 15-step strategy demonstrated 
sensitlivity toj problem differences. Implications for design 
methods researjph/ are discussed. - - ' 
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CHAPTER. I - INSPIRATION: The NeeU For Design Methods Research 

' ■ • ■ 

' ■ ■ . ■ * 

lntrQducti.on: 

Design methods, the working techh:jLques and procedures 
used by professional designers in. fields such as architecture 

«. * . o 

i i 

and eng»ineering, have been regarded with interest in recent^ 
ye^rs as one avepue through which the efficiency and etffec- 
tiveness. of design /activity could be imprpved- This interest 
has* been proitipted^ b|^ a growing awareness that past, traditional 
and intuitive methods are proving inadequate to deal with cur- 
rent problems of social and technicaT ^complexity . • However, the 
development of feffective new methods has been friistrat^'d by 
the lack of • appropriate controlled research in the "areas of 
coirtplex problem-solving and design methods; This study repre- 
sents an attempt to provide a more rigorous examination of se- - 
'lected design methods and their interaction with thd design pro. 
c^ss. Specifically, the ^emphasis is on methods used in Environ- 
mental Design, a field representative of a broad-range of 

design-oriented disciplines. - ' ' ^ 

* . . • . * ■ 

ENVIRONMENTAL DESIGN 

■ • • 1 , . • - ' 1 . ^ 

Environmental design is an ismerging discipline which [ 

encompasses Urban Design., Architecture, Interior and^ Product 

DjBsign. This dj.scipline links the previous loose community of 

designers and design activities within an overall ' framework 

which permits approaching anji solving problems at all levels 

of human environment (Maldonai^do, 1966; Malone, 1960; Musgfove, 

1966; Van der Ryn, 19 66) . As such, -by definition ^nd inclina- 

. . - ,11 ■ ■. ■ { ■ 



tfion, it is involv^^d in^ the interdisciplinary concern for def 
' sign, metfiods . Th±s concern has been described by Simon (1969) ' 
as the "science of c^sign^" represented by. fields auch as engi- 
neering, architecture, business, law, and medicine*. Simpn 
identifies the common aspects^, of these fields under a definir 
tiori of design as "devising courses of action changing - exis^jiing 
situations , into preferred ones". Environmental design can 
therefore serve as a typical example, current dijEficulties 

and potential method utilization similar to other di^9ip'i4ne^^ 

^ A, Environmental Diesign Res^^p^^ > ^ 

. The area 6f^ responsibility, tor envirxi/niUBntal design is 
"iny^nting physical things wh^gh disp'lay new order, organi- 
zation, and form, in response to human function '■-(Alexander, 
1969) . Within this definition, -physical thi'^ngs" can involve 
any aspett of the built environment "^^^^^ 

..V. " • " -* O " -■ V 

spoon" (Maldonaldo, 1966). Particular, aspects of human lunc- 
tion have beeri defined as "freedom of choice, physical 4pd 
mental health, and a richness, and depth bf 'human experience" 
(G^^es and Spring, 1967).- The critical nature of the field* . 
ris apparent when these definitions are.appliisd to specific 
environments .associated with housing, educational and health ^ 
facilities, work places, transportation^, et.c* / c 

B. Environmental Design Difficulties "V ' * ^ 



It should be apparent to most observers of modern '^so- 
• ciety that environmehtal designers have not been adequately 



* solving prqjblema in their area of responsibility Some de- . . 
signers have noted this general inef f ectiv'eness in terms- of 
• "nightjnare- of the cities"' CWachsman in Banham, 1965); "discom- 
• forts heaped on us^by ill-design" ' (Van der Ryn, 1966); "misman- 
ageftjefnt through anti-design" (Maldpnaldb , 19'65) These short- 
coming-s have been related* to the "quantity", "complexity", and ' 
"difficulty" of current problems "associated •with changing 

• f 

soci'al patterns and culture (Alexander, 1964). Other causes 

. • . * * 

» have been identified in terms oi new production capabilities 

which can reproduce "^Imost ahy desired shape in almost any 

^material" (Pye, 'l9£4) . In pj^iTticular^, it has been observed 

• ' ' v< ■ . V , . " 

that changing concjitions haye forced the designer to assxime 
a decision-making burden far more compTicated than in the 

, . ^ast, where options were severely limited, and design? were 

^ •> ' ■ » . ■ . 

frequently based on centuries of adaptation and developjnent 

^ (Alexander, ^1964; .Archer, 1960). • • 

■ * V • ' * ^ ; '. ' ■ 

: C: Environmental De sign Information ' . ' " 

The difficulties experienced by environmental* designers 
in decis^^n-making^ have been traced in 'part to the^ incr.eased 
load on information handling capabilities Vithin the design r 
" process .t^ In 'general, the , information load is an inevitable. 
, result of technological ''innovatit>n which produces a doubling 
of available IknQwledge every' 10-15 years (Price, .^^^) . For 
environmental designers the. situation has been described as a 
"growing bd'dy of knowledge and specialist experience" with 
problem statements; becoming mor^ comprehensive in scope and 



] 



-more complicated N^n nature- lAlexander,, 1961; Cheetham, 1964). 
In addition; the information available for environmental de^ 
signers 1 has been referred to as "hard to handle", "diffuse"," 
and "disorganized" (Alexander, 1964). The situatioft is "^e- 
.scribed by one author as "it is difficult to find 'even modest, 

• accurate, and' systematized information on any aspect of archir 
tectural practice" (Passonneau, 1965)! Without proper methods 
for coping' with this information burden, the designers', ikforma 
tion processing capabilities are of ten - overloaded (Alexander, 
1964) . ^ ' j 

p. ^^Environmental a-Design Methods 

The present recognition given design methods in the • 
field is particularly significant in- light of the f>ast re- 
liance on- traditiohal approaches, characterized by. an empha- 
sis on intuition and practical experience (Cheetham, 1964; 
L. Jay, 1963; Maldonaldp, 1965). The awkwardness of these 
methods is suggested by Archer 19 70) : 

Traditionally, the designer 'has^ worked intuitively . . - 
for the most part, studying, his' brief,-, scanning the 
, . evidence, ruminating upon in>pli cations , and sketch- r , 
^ - ing ideas without exercising much conscious control- 
over' the activity of. desigh decision-making itself. - 

The concern for altetnal^ive , 'more 'effective "methods has been 
expressed in terms of the need to "emerge from cloudy pragma- 
tism'' CMalone, 1960); using "a methodical modug pperandi" ' „- 

' ■ - . * ■ , ' ' - 

(Corki-ri, 196 8); a more methodical'arid logical, way of working" 
(Thornley, 1963) ; and establishing "sound, heurit^tics for de- 



sign problem-solving'' (Kitts,^ 1965). In ^articular, an emphas 

has been placed on "systematic" ways of working, which could 

contrql problem information and lead to more effective results 

(Ar(jher,;a965; Blake, 1962;* Handler, 1964; Jones, 1963). 

' DESIGN 'mJITHODS . ^ V • " 

Design methodsV' dfe^pite their ijnportance , have been' in- 

terpreted in a number of dijfferent ways, with'out any de£ini~ 

tive statement of the concept. In general terms, -"method's" 

usually suggest a mental procedure or set of procedures, 

which provide- an approach^dr framework to guide the\ design- 

er's thoughts, ot actions (Eder, 1966; Gutman, 1966; McCrory, 

1966). Synonyms for design rtjethods include: "strategy", 

"subroutine", "tricks of- the trade", "rules of' the game" 
. . . ' ■ Ck- \ ' . 

(Ainarel, 1966;. Eder, 1966; Newell et, al. 19*59 ; Thomas, 1965) ; 

Qualificatioris on the use of methods suggest that they are not 

to be used as a recipe', followed step-by-step, but rather are 

to be used flexibly, adapted to a specific; si tijation^, as an ^aid 

not a replacement for . intuition ^(Archer , 0,970; Gutman, 1966; 

Krampen,* 1965).^* / . " ' 



A? Utility of Design Methods 

The utility of appropriately developed me^ods has^ been 
described in terms of the potential for aiding/ the designer, 
particularly in the area of information processi^ig. In overall 
ierms, it is suggested that a specific approach can ijifluence 
the level, type, and sequence of information usage within the 



' design process (Mitroff , 1968; Ranstrom and Rhenman, 1965) . 

For one. method the advantage!^, were described as: 

aids the consciousness to focus on objectives, 
analyze problems., ^ and select, reject-, and organ- 
ize data in those patterns likely to pro'duce 
subconscious closure and interpretation (C. Gregory, 

^19 6 7) . ' ' ' ' • ' ^.^^ / 

* - 'In the area of visualiz4tiori, specific reference is. made 
to "expressing the Jprief in \ex:ms that make possible a systeiju-* 
atic method of designing (M. J^y ,« 1967) , and "a way .of pre- 
senting design prob^lems that does make them easier to solve". < 
(Alexander, ^1964) . • • • , 

processing information, methods, m^y assist in direct^ 

ing the search' for a solution and evaluating alternatives , 

r ■ ' ■ 

' ■ • • . • /■• • 

with a 'minimum of error , redesign^ and delay (Amarel, ,J.966; 

Jones, 1963; Miller et al.,^ 1967) . -.Simon (1969)' provides one 

' • • ■ . ' ' * .■'*■•■/■/■' . ' •/ 

interpretation of method advantages as : . 

manageiAent . cif the** res^ources of the des igner 150 that 
his. efforts will not be dissipated unnecessc^rily in 
following lines of inquiry , that prove fruitless. - 



The potential outcome of these aids has be^ ^iven in 
terms of "the evolutioriary>development:* of * successfully im-" 
proved solutions^'.k^Roe et aP, , 1967), and more imaginative 
and advanced designs" '(Jones, 1963). 

B. Availability .of design Methods ^ > 

Ai£erna'l;:|.ve design methods have been developed' in recent 
ye^rs by practitioneirs in environmental design and related 
f ields . in an effort to realize the -^p-0±^ntial advarftages of 
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appropriate methods . ^5 Majo'r methods developments have bee^i 



attempted by Archer (1970); Asimow (1964) ; Nadler (1970); 
C. Gregory (19 67) ;* Roe et al. (196 7) . Symposia and collections 
of articles / on diffei^ai^t techniques ^nd meth>|^^^^,have 'been ^ 
edited by Jones and Thornley (1963) ; Zwq^cky and Wilson (196 7) ; 

S. Gregory (1966); and Moore (1970).. Comparisons of different 

\ \ ' t _ 

methods have been made by Alger and Hay (1964); Whitfield. 

(1968) ; ' Guilford (0.967) p and Joner^ (1966) . Engineering design 

■ > * 

methpds have been described by Buhl (I960.) ; Dixon <1966) ; ^ • 
Pare et al. C19 63) ; Roy lance (1966) ; Wilsoii (1965)." Specific 
des ign technique s have been developed by Gordon (1968) ; Alex- 
ander . (1964) ; Alexander et al. (1968) ; Goodet^and j^achol;' fl967) ; 

■ "v. ^ ■ . . • ■ . • , ■ • 

Starr (1963). Computer-aided design has b^eh discussed by 

Campion (1^6 8) ; Fetter (1965) ;Siders (1966) 

■ . . df ; ■ ^ 

C. Developing Design Methods * • 

The majority, of the available methods , however , are 
empirically based, the result of individual experience. ^ Such* 
meth6ds have been described 'as "applying their favorite prin- 
ciples to the design problem"/ (Gutman, 1966) / land making 
explicit c" the practtces that discerning people seeit}/to be 
adopting" (Archer,. 1970). Even when these methods'^ have been 
adopted or adapted from other disciplines such as philosophy, 
s oci%T • scie nee econ computer o s cience , * managemen t 



fScienc^, arid engineering (Barnett, 1966; S. I^regory, 1966; 
M. Jay, 1967) , th^e methods have^ themselyes 'of teA been empiri- 
cally based, and t:heir application has ustially not been systema 
tically determined (Studer and Stea, 1966) . Th^re appears to 



V ' . k ' . 8 

(> ' . 

have been little, if. any, attempt to develop design methods * 

i]^ a controlled scientific manner^, permitting a rigorous 

' analysis and evaluation of their effiscts. This aspect has 

been summed up by: • v . . 

Definitions of "better" methodologies are meaningless " 
^ unleiSs 'we are told in what r^espe.cts a method is better 

than pther methods (Kaiser, 19)52) • 

D, Design Methods Research . . * . * ' 

» > It •'Should b6 apparent that the only way to develop design 

^methods beyond their present informal level, is to follow .the. 
course taken by' other disciplines in mbving from an empirical 

■ ' / ' \ ■ ■ I ' ' 

\ to a theoretical knowledge base.- This cbncept has *been der- 
\sc];:ibed in terms of ijivestigating the j nature . of , the desi'^n 

- J , ■• I ' ^ ,•/ .'■ 

process and determining the ways in .which methods/ could be 

appropr irately applied in each phase of design activity 

(S, GregoryV 1966;jKitts, 1965^. -The sqope of such an inyesti- 

gation has been suggested by De Grootj: (1966): 

From experimental evidence and ^nalysi-s, it must be^~ 
possible * to develop an ordered classif legation of - 
human mental operations to cover every "methbd^ 
/ trick/ heuristic^ trans formatiprji that may be instru-' 

'*imental in productive thinking, i V. * 

^ The type and level of research Which is needed is quite 

extensive, but it should be apparent that in a ques t . for valid 

. ; < 
"systematic" methods, that equally^ rigorous means bf investiga- 

tion should be eimj^oyed. .The significance of such re'search * 

activity has been described in the strongest , terms by Simon/. 

(1969) : ^ 
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The professional schools will reassme their 
professional responsibilities just to the 
deg^^e^that they ^can discover a science of 
desigfr; a body of intellectually tough, 
analytic, partly formalizable , partly efnpir-r 
ical, teachable doctrine about the design 
process . . 



CHAPTER .II - INVESTiGATIO^I: Approaches To Design ^Methods 

Research .V * 

■ ' ■ ■ . I 

Introduction: , /l 

* ' Design methods research> on ^almost any aspect of design 
process or metliocis, must be bailed on the study of design prob- 
lem'-solving, the act of (Resigning. Previous experience by cog- 
nitive psycholbgists has indicated great difficulty in studying 
human problem-solving behavior. Some techniques have been V 
developed, but these involve a number of s^ort-comings, While 

there have been scwe studies in design-related fields, these 

' • ' ' ' . ■ • . 1 • • 

have not -dealt with the details of complex 'problem-solving 

nqi^ have they attempted to use 'design methods as an independent 

variable. The purpose of this study is to demonstrate a new ' 

approach to design methods investigation. This approach is 

based on systematically ^compari'^g alternative design methods, 

using a research technique adapted to the identification of the 

impact of methods oh the design process » 

1 • . . . ■ . ' s • 

DESIGN METHODS RESEARCH 

The observation of the workings of the design process 
poses particular difficulties since by tradition and f)ractice . 
most desi^h activity consists oi subconscidus or covert be- ( 
havior, with m6st 'reasoning and critical decision-making re- 
ma ining within the mind (Eder,. 1966; Thornley, 1963). in 
most cases the only evidence available is the solution ptesenta 
tion and a limited display of noteg / sketches, or calculations^ « 
(Eder, 1966) . While it may.be possible to make some general 
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observations" from this evidence, theVcritica;. details remain • 

" ' ■ • . '( 

out of reach (Bloom, 1950; John and Miller ,. 1957).. The basic ^ • 

difficulty has ^been described as the "problem how to observe 

and measure the <process^' and refer^ce is made to tne "method- . 

ological difficulty in externalizing the process ^in an opera- 

^tional way" (John and Miller, 1957) . ' . ^ 

A . ' Research ^ .T^echnic^ues " " - ^ / , 

• * Some techniques have been' developed in attempting to- 
m^ke the problem-solving processes observable. One authof: , 
.has described thes.e techniques' as means 'of "bringing . the covert • 

-processes- into, the open" (Forehand; 196^) . .ThB use of the ^ 

- ^' . . ■ • ^ > ■■ ' ■ ■ . 

. technique has be'en referred to as "exploring* the process . 

itself - investigating the subject's thoughts, feelings, and 

- reactions - as he attacks the problem • (Bloom and Brodel, 1950) . 

) Retrospection 'represents one technique wh\ch has been 
used either alone or in combination with other methods. This 
technique usually consists of an interview or questionnaire' 
after the problem- solving session, in an attempt to recall and 
reconstruct the processes which were used . (Clarkson and Pouiids , _ 
19£3; John and Miller, 1957; ^Mitrof f , 1968). Due to diffi- 
culties with memory and reconstruction, this technique is riot 
'too reliable, but it can.be useful for idenrtifying major 

occurrances and subjective observations. \ 

■ . • - ' , . * • ■ ■ ' •' 

• Introspection is an alternative technique based on eliciting 
information from the subject while he is in the. .process of ' 

" ■ - 2:1 ■ ■ 



, . ■. . 1.2 ' ■ 

'Solving a problem . ( John |ind Miller, 1957) . ^The record^ genen- 

ated lis i.ag this technique is usually referred to as a "protpcal" 

.(Newell and Simon, 1961), which is intended to represent a pic- 

ture.of the cognitive process used By the problem-solver. 

♦ • • • » ■ . ' •* * ' 

Introspection is usually 'preferred over retrospection as pro- ' 

viding a more literal accurate picture of the siibject's behavior 

■ • \ 
^ ■ • \ 

(ClarKson ^nd Pounds, 1963) j,, . . . ' . 

Thinking Aloud is a particular technique for generating 
introspective^ protocols, based on subject verbalizations yhrch ' 
are recorded on magnetic tape as he attempts to solve or think- 
through a given problem (Clapkson and Pounds, 1963) . A sgeci- 
f~ic description of the application of this technique ,is given' 
by Miller et al. (1969) as: 

M - Thinking aloud r-equires merely that ^ne person talk 
while he is working, that he should comment on what 
he^is doing, what he is looking for, what his in- I 
tejxStions are, what objects or relationships catcji 
his attention, etc. • . 

These authors have cited the popularity of the thinking alou^ 

.technique "used by Binet/ Duncker ,. "^Glaperede , and many others", 

and indicate'^ts advantages, "when the method is used intel- 

ligently arid conscientiously, it can provide a tremendous^ 

amouj^t of information about the detailed process, of ^thought". 

However-, this method has some drawbacks which' will be shown 

■ , ' .. ^ •• • 

later in the discussion. i 

B. Problem-Solving Research . ' 

A large number of studies have been completed, using the 

* •../■■. 

above techniques. However, most of the work has had minor 
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impact on design methods research since' the emphasis h^s. been 

(<. ' ■ . ^ ' - ' 

on basic "problem-solving", using relatively simple, highly 

•■» ■ 

constrained test problems. The early work is characterized 

I. . ■ ■ ■" ' . ■ 

by Maier^ (1931) and Duncker U9'45) who studied sub jects solving 

physical and mathematical logic problems . Their emphasis v^s 

on demonstrating the formation of '/!^pe\pif ic conceptual relation- 

•' . • , / \ \ 

ships as the keys to solving problems. This approach and that 
■ ' .. ' • . y 

ofNrelated studies was^ based oil concepts. ^related to Gestalt- 

psychology, and similar work investigated mechanisms relarted * 

to "deductive" and "inductive" reasoning, and "concept Jforma- 

tion" (Was on and Johnson-Laird, 196 8) . ^. - . - ' 

: ■ , ■ ■■ ■ " • • ' ■ ■■■ . ' •.. :.{■ ■ 

An iftformairion processing approach to problem-solying 
was taken by Several 'investigators typified by Newell, Simon/ 

• ' . : •* --^^ 

Paig^e, and others (1958, 1959, 1961, 19§6.) . This research 
.relied heavily on thinking^. aloud protocols generated by 
subjects involved ' in/ somewhat more complex tasks such as .sym- ... 
bolic logic and algebra word problems. '|'he outcome was a . 
comprehensive analysis of these problem-solying activities . , 
with, the development "^of' information processing* models bfeised on 
"objects"- or problem states , transformed by "operators^^ which 
w^re^ "apiplied to certain objects to produce different objects". ' 
For the rela^tively simple, essentially mechanical, problems 
being ^studied it was possible to represent hurtan groblem-solving 
behavior using computer programs such as the LOGIC Theorist; GPS 
(General Problem Sol^j;^er) and STUDENT. , 



^'^^searchers rn engineering and environiriental design have 



a'ttemptecJ invostLgate aspects of complex problem-solving • 
However, these studies have tended to, be more general and open- 
endeA than the controlled .studies of*>basic problem-solving. 
Typical studies were performed by pixon (196'4;i , Ranstrqm and 
Thenman (1965), Frischmuth and Allen (1969*) , consisting of ob-r ' 
servatibns of work dcjne by ^^racticing en^ineer ^Qn their real-^ 
world -probj. ems , using notes,' tapes, and intervie^ . The results 
of these studies^^were general models of the design process * , 
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expressed as " f Jtowcharts " (Frischmuth and Allen); and "need", 
"engineering", and "product" dimensions or development stages 
(Ranstrom and, Rh^^tt^an) , *v r ' ^ 



Similar studies were performed by Mitroff (1969) alnd 

Reitman (1970) using more detailed thinking aldud protocols 
. » ' • • 

-to investigate ^ specif ic^ aspects such as designer's attitudes 

/^inid other subjective factors; and interactiohs with "open- 

%^ ' ^ f ' ^ ^' ' • 

vended" -or "ill-defined" inform'dtio'n, respectively. The re- 

suits of these investigations suggested that even in relaltively 

"constrained firoblem-sblving situations there iwas considerable 

dependence x>n individual interpretations. . 

Particular controlled design studies were conducted by 
Meister and Farr (1966, 1967) using subject groups of 10-20 
engineers working on pre-specif ied ins'trument panel or console = 
design problems. The experimenters provided design specifica- 



tions and background information; including data on human 

• - ■ • ■ . . 

. f actoris req.uirements , and employed obser^i^ation of .^sub jects at ^ 

work, 'subject comments while working, and in-d^pth retroapec- 

tive interviews. These techniques were used to determine the 

•extent of human factors information utilization. The results 

indicated that the sub ject-desigjiers generally used a^much more 

superficial problem analysis than had been anticipated by the 

• \ ' ' . , ) 

experimenters, and that subjects usually ignored hifinan, factors 
data presented. in the typical handbook form./ 

Research s'tudies in environmental design have been less. 
numert)'us, but fojlow the patterns associated with other dis- 
-ciplines. ^ % typicafl slinplified experiment was used by Krampen 
(1965) n with classroom students working on a^ graphic layout . 
problem with hrghly constrained problem specifications. Think- 
ing aloud protocols were- generated and used^ to <3emons trate ' to 

• * ' . ' ' ■■■■ \! ' ..." 

Students basic steps in problem-solving,^ such as "definition," 
"solution", and "verif icatioji" . , " - r 

- • ' ' . . ' ■ .. 

A simiolar experiment was used by Eas'tman ' (1969). with a'^ 
large number of subjects solving a ^'space ^plarijiing" problem , 
uaing a highly constrained format with specific ruies^^f of 
location of elements.' Relatively detalledf thiAkilng ^kloud 
protocols were used in an information processing Analysis, 
identifying "operators" 6r transformation^ and "tables of 
cojinections" which indicated different applications of the de- 
sign.rules. The work was also used to demonstrate "search 



strategiea'Uidentif iQd by Newell and others as "generate and * 
test" (trial and error) ; "means-end analysis";, (sequentially 
applied constraints) ; and "planning" (solving problems by 
subsystems) . - ; * 



^ Another study by Eastman (1970) used a more' open-ghded 

problem-solving t^sk, the design-of a bat hroo m layout , '-with 

. . ^ ^ : ■ r . . 

minimal requirements and an 6mphasis/on individual insight 

\ ' ' ■ ■ ■ ■ , • -» . 

and* past experience. 'Analysis of problem-solving was based , 

* ■ • * ■/•I*' * ■ ' 

on thinking aloud protocols * and graphic . sketches . ' Results 

indicated a correla^tion between the foojcn of the requirements 

and thefr representation by subjects'; ^and cited the superiority 

of • dependence oh information stored in th.e memory over i^quire- 



"^ments giyfen with* the problem. 

D. . ReseArclj Implications 

* . ■ • • • ' . ' 

i ' ''' .\- ■ ' ^ ■-" ' 

The major observation from the review of- related studied 

is that there has heak little or no research related • specif i- 

cal-ly to design mjsthods. Where research^h^s been re^^ated^, to 

design, the emphasis has been* on , studying how ^ design prapti- ' 

tioners use trad;Ltional, intuitive methods rather than cbnside] 

ing the effect of alternautive improved methods. l*he. work dofes 

suggest the possibility of observing viand analyzing design be-^ 

havior, but it is necessary to go beyond 'the consideration of 

limited problems and specif ic, mechanisms to consider design' 

■ V ^ ' ■ ■ : .. . 

problem-dolvihg in greater .detail . In particular, it is nerce^- 
sary to begin filling- in the specific behaviors associated 



with the major stages or overall approaches used by designers, 
and to begin describing how different types of information 
interact throughout the design process. To accomplish this, 
design methods resear^ch must not only^ test" and cdmpare . alterna- 
tive methods r but must also de'^elop tools for studying and anal- 
yzing complex problem-solving behavior with a rigor which will 
provide a sound theoreti^cal basis for understanding the me- 
chanisms and interactions which are invglved. ' 
» ... 

In another aspedt' of the related research, the "jthinkihg 
' aloud" technique doe| not seem to be effectively applicable 
to design methods research. This technique has been criti- / , 
cized in terms-' of^: • a.) its potentially inhibiting effect on 
problem-solvers; b)* ''the difficulties yith confusing termin- 
ology, ommissions, and inaccurracy; and c). the effort /iji 
transcribing lengthy problem-solving sessions (Bloom and 

E 

Brpdel, 1950;. Miller et al., 19.69). An alternative technique 
is required which could offset these disadvantages and add 
greater control in terms at 1) accuracy in determining sequence 
and chronology o£ events; 2) standarized coding of information 
* and responses; 3) identification of relationships bet^^reen 
elements in the process (Bloom and Brddel, . 1950 ; De Grobt, 
1966? Forehand, . 1966) . 

' PRESENT RESEARCH APPROACH ' ^' ^ A 

The purpose c.t this study was to make a pilots attempt at 
9_.design methods research, * using design methods as tke ma^or 



ifesearch variable, in an experimen:tal setting designed tq 
generate specific behavioral data about the effect of meth^s 

.on the dfesigner-^ubject *s problem-sgl'Q'ing process This 
attempt was" based on implications of the related research 

,and.the nature of design methods as previously described. 

ji. \ Comparing Design Strategies • 

Design Strategies ^ere selected as methods Tyhos*e effec- 

'tiveness could be .usefully compared in^terms of their promi- 

'nent role xn the design methods literature, and their impact 

throughout th*e problem-solving proce;5S. Strategies have been" 

defined as "plan of attack, methodology, or approach", which 

xs char acif^^rx zed by!? 

An rteratxve eerxes of steps defxnxng the direction 

or pat.tern' Of action to be followed by an individual or 

group 'in achieving a purposeful activity (C. Gregory, 

1967) . ■ • r ' ; • . - 

As xndxcated earlier, strategies were identified • by extrapo- • 



Eastman's work. I'h^oretical ^discussions have occurred in the 



lation in several engineering design studies/' and in some\of 

work of Nadler (1970), Jones (1966), Archer (19 70) ,' and 

C. Gregory (19<67) ,> among others. 

.•'\..> • . . . . * ■ - ." ■ 

Strategy differences have been described by C. Gregory ^ 
(1967) a,s a result of a survey of strategies which demonstrated 

* ' r ■ . 

variations in terms of: a) specific steps mentioned by sonfe 
authors and omitted by, others ; b) combinations of s^teps under 
mof'e general terms; c) ° ^^transpositions in the order or sequence 

: , ■ • 28 . ' ■ 
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•of steps; d) differences in the total number of steps. These 
differences may lead to different strategy impacts as described 
with the specific experimental hypothesis on pages 38-39. 

An important aspect of a, comparative study of design 
strategies is the opportunity to refute current misconceptions 
that either strategies have no effect on^the design process or 
all strategies are essentially the same. This latter attitude 
is^suggested* by Eder,(1966) referring to a given strategy 
example,^ "all methodologies so far .proposed fit into the de~ 
sign sequence outlined above'**. 

B; Complex Design'^ Task ^ 

'The observation of -a' subject .working on a relatively^ 
complex design problem-solving task was determined as the 
-most appropriate setting 'to compare the effectiveness of. ^ 
design strategies . This determination was based qn the need 
to obtain more controlled data in general on ^omplex problem- . 
solving, an* need for sufficient scope to determine strategy 

effects on a'^range of -behaviors within the desJran process. 

/ V . - ' 

For this research such a task would involve the) design of an 

environmental setting such as a room or similat space* accomr 

* ■ ■ #- . / 

mod^ting a range of activities related to a ppecific purpose 

s • _ 

or function. This task would be considerably more complex than 
a simple logic or layout problem, and would have a SQmewhat 
broader, scope than Meister and Farr's co^nsole problem of 
Eastman' s -bathroom. The need for research at this , level' hasc 

• • 29 



been acknowledge by Green (196 6) as an ''interest in -complex.^, 
or as they sa^ 'rich' experimental settings s^o that the com-.- 
plex structure of man's behavior can be displayed". 

C> ' Observation J^echnique ^ " 

'* An experimental technique which generates data-.on the sub- 
ject's use of "-informatron" within the- design' process was 
identified as most appropriate to de'sign methods research. 

V • , 

This was based on thq understanding developed earlier that 
information processing is considered to be an important element | 

in design problem-solving and a? major area affected* by design 

*i • ' , ■ 

methods. Such^ technique would develop an information pro- - 

*' . . 

cessin'g protocol", a 'record o;f the individual informatioil items 
u'sed in problem-solving, genera*t^cj as the information was '^.etu-ally 
.used, and i'ncluding manipulations and relation^ips associated ' , 
with specific i^nformation items. * ^ 

The key to the technique would '^be an information bank, .a. 
collectibn of inform'^ation Ln^^rious categories covering all 
aspects of the problem-solving task. ^ A separate information - 
bank 'could be 'pre-specified by the experimenter for each 
design problem, and subdivided into separate items . The items ., 
could . then be printed on individual cards and physically 

selected and manipulated by the. sub ject ;^as an ana:log to personal- 

■ ' ' ' ■* 

notes or mental manipulations. This technique was intended to 

• . • ' ' 

provide a relatively natural working medium, simplifying the 
processes of information search, retrieval, and rnanipjulation, 
and providing , for a direct recording of design behavror m 



terms of - information patterns. Th±s would be in contrast to 
the i\ndirect thinking aloud approach which '^i^qncentrated on 



subject perceptions . . By making the process literally obs'erv- 
able through the use of sets of information cards,. it should 

• . -^V ■ ■ ■ ■ ' ^ •■ "'-^ ' ■■ • 

be possible to obtain ^ standardized/ highly accurate, highly 

\ ■ ■ \ " • . - 

■ \ . . . > ^ I . 

reliable record,, with a^ clear reading of sequence , duration,, 
and- relationships 




' While this technique was distj.nctly different from the 
roaches Wsed in most of the related' research, it was suggested 



approacnes ^ 



em- 



by controlled inf ormatigh formats used in man^ of the probjLe 

solving st.uda^es. Although the use of inf ormation • items ^. as a 

\ • - • . ' \ ' ' ' 

design me di\am\is somewhat ^unusual, this approach has been sug- 

■ _ \ ■ . y ^ ^ . J ."■ ■ 

ge.sted by seve3s;al design practitioners, who have -recomLmenHed 

■ ^ ' \ • ; ■ • • ■ . -, 

r'^cording and manipulating informations-using index cards or 

slips of paper (^rnette, 1967; C.."'' Gregory, 1967; Jo.ne's , 1966) . 

In a theoretical s^nse, information protocols are consistent 

with the view of th^ design process described by several 

* -. V : .\ ■ • ■ • ^ ; . • . • ■ 

authors as an informq.ti6n image being continually developed 

and refined in progreS,sing from an initial problem definition. ' 

to a final' .Solution' (Bii^nette , '.1967 ; Gregory 1966 Fore- 

hand, 19.66; Krampen, ' 1965^ . Mii;Ler et al. ,•■ 1969.) . 



E> Experimental^ Design \ 



A 3x3 factorial designV compa^ring 'three design strategies', 
used . to solve tho^ee problems;; with three "sub je,pts using each' 
strategy, Wc^k determined as ari appropriate scale for; this pilot 
research, providing some variety and comparison -of differences . 
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within the resources ^available to i^he ' experimenter . This 
'offered particular advantages since hy comparing three strate- 
gies i^ was not necessary to use a separate control group 
(using no strategy) . By comparing strategy effects over 
three problems it would be . possible to check on the generality 
and consistency of the strategies, with some control over 
di'^f f erences in subject background and experience ^ which might 
have favored any given problem. It was determined' that each 
s'ubjfect should use the 'same strategy for all three probl^s, 
because of the likelihood of confusion and Gontami.aation if 
subjects were asked. to change strategies between problems. 
This design also permi;ts a statistical analysis of variance 
to determine effects of strategies , problems, subjects within 
strategies, and strategy -problem interaction. 
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Figure -2-1 E:>tperimental Design ' ^ 

E . * Subjects' . / . " 4 . '■ ■ -c- 

' Advanced environmental design students were initially 
identified as a . general .sub ject typ^ appropriate to this 
research.. This was based on the Assumed need for ^. subjects 
able to solve complex des-igfn prpb|lems in a reasonably brief 



- period of ^time/and the relatrVely ^large trme coinmrfetment: re- 
quired for Orientation ari^ for three problem-solving sessions 
Students also seemed desirable in t^rms of their availability 
i on campife, their generally high le^vel o£ interest and motiva^ 
tion,, and their relative flexibility in working habits as com- 
.parfed with prof essiQnals . ' V 



ERIC 



33 



CHAPTER III - INSTALLATION: Development o,f Experimental/ 

. ► . Materials . . ^ ' 

Introduction: ' 

Specific experiunental material? , ^*^oblem types; test 
strategies, information banks,, and apparatus were developed 
using controlled systematic procedures. These procedures 
were used to provide the best possible tools for this research, 
with a comprehensive description of ^ the development process 
for use by other design methods researchers. / , 

i . . * 

TEST PROBLEMS 

Probiem-sblving t^sks were selected using criteria devel- 
oped and applied ^to problem a*lternativQs through a series of 
questionnaires submitted to a committee of coi^isultants (the 
experimenter's faculty advisory committee) . — • 

Al Establishing a Problem Area 

'The range of , problem alternatives was limited- to facili- 

■ V . • ■ ■ / ; .. ■ 

tate t:rit,eria and problem generation. Environments associated 

with college classroom buildings were selected in order that 

their familiarity to subjects could reduce the tinje required 

'for problem definition and orientation. ; Such environments 

also: a) are familiar, to the experimenter; b) * of f ej^ a vari-- 

ety of settings; and c) havie a likelihood of exis ting informa-• 
. ' ' ^. ■ ■ . ■ • ■' X- •■ V * • ■ 
tion and expertise available ^on the campus « ^ 

' ■ ■ ■ : : ■ ^ ' ■ " ■ . . . . ' ^ , . 

B. Advisory Committed 

The* advisory committee was involved because of the breadth 



. ■ 25 . ■ 

of expertise represented by the following academic departments 

... ^ 
and subdiscip lines : K ■ ' ^ . 

1. In^ustri^l Engineering (design methods and systems) 

2. Industrial. Engineering (human factors, ergonomics) 

3. Psychology (engineering and perceptual psychology) 

4. Computet Science (information, systems) ^ . ' ' 

5. Urban Planning (architecture and environmental design) 



These fields ..includf aspects relevant to environmental design 

X •• • / ■ 'T ■ - ' 

and the con§:lderati6n of research tasks and experimental de- 

signs. Faculty members should also have a sensitivity to 

facilities for higher education. 

: C. " Delphi P'rocedure , , . ^ ' 

An adaptation of the "delphi procedure" (Quade, 1968; 
Reschery l§69).used a series of questiortnaires , permitting 
the CQjnmit tee members to no^te the responses of their colleagues 
as^ an additional factor in making subsequent decisions. This \ 
allowed the committee to participate in a simplified dialogue 
and search for a consensus without physically meeting. Two 
questionnaires were used for critdria selection and a final 
questionnaire was used for applying the. criteria to. problem 
alternatives. 

D. Criteria Review Questionnaire 

'The first questionnaire priesented the committee with^a 
tentc^tiVe list- of criteria for their review and comment. 
Criteria were generated by the ^experimenter based on* the needs 
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of the experiment and. the characteristics of problems which 
might be evaluated (Appendiji A-1) . The specific ctiteri'a 
titles were as follows: " - i 

1. Comparable tp other Problems ' ^ / 

2. Environmen-^al in Character 

3. AVailability-'^.of Background Information X 

4. Motivates Subjects . ^ ;3 t 

5. Unders tandiable 'to Subjects . 

■ ■ ' ) ' ' . 

6. Permits wide range of /Solutioh Possibilities 

7. Reasonable to Expect Solution v/ithin Time Constraints 

Criteria presentation 4-ncluded: a) a brief revie w of th e 



delphij procedure; b) an explanation of the initial choice of 
problem area; and c) a simplified description of the role of 
test problems in the experiment. A response sheet provided 
for conunents on the individual criteria and on criteria that 
might have\been omitted. The results affirmed the proposed 
criteria 'and suggested minor modifications which were included 
in a* revised criteria list for the second c^questionnaire 
(Appendix A-2) . * . 

E. Criteria Rating Questionnaire * 

On the second ques'tionnair^e the committee members ^were 

asked to compare the different criteria^ and estimate the ^ 

relat^ive differences in importa'nce. ajhe revised crijl£eria 
were presented to the members with an expanded set of defini- 
tions to. aid in understanding (-Appendix A-2) . The response 
sheet required anci indicatidn of differences on a linear scale 
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as an ai4 in visualization and a means of insuring compara:- 
bility results.^ The results »bf the questionnc^ire are pre- 
sented in Table 3-1, showing the ratings for each t:ri1^?x.on 
numerically translated from thexr linear p- ^itions. ThQ rat'-, 
ings were frequently i^icohsistent , and in'- several instances | 
member^ assign'ed v,ery aifferent ratings to the same criterion 
(A, D, G) . Given the mixed results, it was not possible 
to confirm or • re ject a^y specific criterion, or to assign , 
any meaningful vefCghting. 



Table 3-1 Responses 'on Criteria Rating Questionnaire' 
Hiumerical equivalents of linear positions, 10 = high) 



CRITERIA 


■ RESPONDENTS 


•aver/^ 




■ A 


b' 


c 


D 




A. Comparably With 
Other Problems 


5.7 




0.6 


3.6 


2.5 


^B. Environmental Desigr\ 
Task 


9-9 


10.0 


2.4 


.4.5 


6.5 


C. Availability of Back- 
ground Information 


9.0 


IG.O 

□ 


6.9 


9i9 


9.0 


D. Motivates Subjects '* 


1.8 




. 8.4 


8.7 


4.8 


E. Time Constraints 

0 


8.4 

.'4- , 




9.6 


.7.2 


6.3 


F. Comparable Within,/ 
- Problem' Area 


6.6 


10*. 0 


9.0, 


7.8 


8.4 


""G. Uniqueness of Prob- 
lem 

i^. : '1 : 


0.6 




5.4 ■ 


9.3 


3.8; 
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Problem Selection Questionnaire' 



in the thir^ questionnaire , a^.s-^lected set of criteria- 
(Appendix A~3). were applied to a aianiber of problem alternatives . 
developed ^by the' experimenter to represent environmental systems 
with "a" range of components or elements to be. provided.. * Prob- 
lems were eliminated from the larger set of alternatives in 
the educational area if they were too complex in terms,, of ^" 
variety of spaces/ activities, or unique technology. The 1 
typical classroom was also* exclu'&ed as being too simple under 
presisnt interpretations and too complex if thoroughly djesigned 
for new teaching 'teeti^niques . The final set of alternatives 
(Appendix A- 4) was subdivided into three different categories 
to maintain the distinction be^tween problems, as sji own below: 

Group I Group II ' , Group III 

A. Faculty Offirre E Dept.' Conference ' I . Campus Snack 

Room ■ • ' Bar 

B. Administrative . 

Office * * F.. Lab Discussion . J-. Dorm. Floor 

Room Lounge 

C. Research Office 

Gi Teaching Discus- K. Departmental 

D. Teaching Asst. s ion 'Room « Lounge 
0£fice 



' " H. Audio-Visual Room L'. Coffee Shop 

A. paragraph description identified the , alternatives, without, 
descriptive titles in order to minimize bias in terminology. 
One response sheet was provided for each of five criteria, 
with cues representing high, middle, and law values (Appendix 
A- 3) . Cue statements were included at .relative intervals 
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along.a Ixnear scale. The ^r^sponse format r^q^ired the ^ 

. - ^ " • ' * i ■• , . ^ ■ = .. 

mtBmbers to rate th^ f our altamatiyes in each group, >-Jvi4:h the 



Understanding that the best problem x«b '^each group yould be 
us ad' in the experiment. ^- n ^ 

The '"results of the third questionnaire are shown in Table 
3-2, again using numerical equivalents to represent linear 
positions. The results were once more varied and contradic- 
tory, and in some cases committee . members completed only a 
S>^prtion of, the questionnaire. . However, the results identi- 
fied several probTems as overall high scorers. The scores 
were used in a further screening of the problems by inspection. 
The final selection^' were made as follows: (I) th^^e faculty 
office was relatively more clear-cut thdn the TA office with 
, a likelihood of considerably more background discussion in the 
literature; (II) v the departmental conference room was more 
appropriate than" the two classroom-types which seemed too ' 
• open-ended, "and the audio-visual 'laboratory which ^e'emed to ' 
b.e technically difficult; (III) ^ the campus snack bar was less 
technical than the coffee shop, the departmental lounge was 
too sim^iiar to'' the conference room, and the dormitory lounge 
was too remote fioom the classroom building. The selections 

^ seem to provide a representative set of problems within the' 

* 

bounds of iihe criteria. - In retrospeqt, despite' the incon- 
sistencies in the qties tionnaire technique, it does seem that 

the moite explicit exami-nation and decision-making resulted in 

ii ■■ • ■ ■■ ' •• • • ... 

a "more rigorous selection of problems than would have been 
likely^ using less formal methods. ■ . - 
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Table. 3-2 Responses on Problem Selection Questionnaire 
(numerical equivalents of linear positions.i iO^'high)'' 



(c) (:.; (c) Cr) 


le) : (i) .(,1) (k) (i.) 


aral-.3.e 
To 

ether 
pT>ot- • 


A 






• 


4.5 


4.;- 




4.5 


'4.5 


4.5 


4.S 


4.^ 






. •• •. y 








• _ 


. • ^ 


. _ >- • P 






R 




1 R 


c 

T 






> ^ 


C) 






R' R 






P*2 


R ■ O 

p.u 


p -n 




1 c-^ 


An C 
1 .. . ^ 


PI 0 








IZl r 


1 R 
1 0 . p 


1Q R 
1 7« P 


IP n 


^. . u 


Hoti- ' 

vates 

Sub- 

0 ^ ^-^ 


A 




.0 


0.5 


2.0 


5.0 


2.0 


7.0 


9.0 


^.0 




6.0 


■4.^3 


B 


2.0 


4.0 




9.0 


9.0 


9.0 


9.^0 


9.0 


9*6 


7.0 


9.0 




C 


2.0 


2 0 


^ 0 




7 0 




0.7 




p. 'J 


r -P 




R n 
p*u 


D 


6 0 




10 0 




Q 








R n 




p n 


R n 


T 








PR n 


^lO D 






XR R 
PP«P 


PC? n 


^1 R 
P 1 • P 


PQ h 


^ Q n 
1 >. u 


■tiiire 
For ' 
/Solu- 
tion 


A 


4 0 


^-') 0 


o n 
y y 


Q 0 


^ Q n 
^.u 




Q n 


Zl ^ 


Zl ^ 


■Q • D 




Zl R 


"R 


10 0 


Q 0 


y • ^ 


Q 0 


Q n 




Q n 


0 . u 




0 . U 


P n 


^ R 
0. p 


0 


u n 


::>• ^ 




^ O 






0 . w 






P.O 


/•P 


Zl R 


T) 








0 • u 




L.J . u, 


0 n 
0 . u 




'in n 

lU,. u 




P«U 


5*0 




55.0 


29r5 


2?. 5 


32.0 


33.5 


24.0 


34.0 


20.5 


28*5 


28.0 


29.5 


20.5 


Aval 1-' 
able • • 
Inf or- 

mation 


A 


•4.0 


4.0 


4.0 


4.0 


■4.0 


4.0 


, 4.0 


4.0 


9.^ 


4^.>0 


4^0 




0 




7.0 


7.0 


6i0 


■7.0 




5.0 


.7.0 


■ 






^.0 


T 


ir'.5 


11.0 


11.0 


10.. 0 


11.0 


9.5 


■9.0 


11.0 


15.0 


10.5 


9.5 


1.4.5. 


CjorriT)- 
arable. 
Within 
:^r,ob- 
1 em s 


A 


9.0 


9.0 


9.0 


9.0 


9.e 




9.6 


4.^ 


4.^ 








13 


9.0 


^^0 


9.0 




9.0 


, e.'p 


9.0 












C 


■n.O 


2.0 


■4.0 


?..5 


4.:3 


.?.o 


4.0 


7.0 


7.0 


4.0 


^.0 




T 


26.0 


20.0 


22.0 


23.5 


22.5 


14.0 


22.0 


T8.0 


18.0 


15.0 


16.6 


16. 5 


Abso- ■ 
lut^e' 


E 


10.0 






. 5.0 






5.0 


10.0 


IQ'.O 




♦ * 


.5.0 




■I'-IO 


' 95 


,102 


117 


11^ 


83 


115 


110 


110 


105 


'102 


84 
-j 



A-E' indicates different respondents ,0n'ot. all respondents 
•answered.' every 'criteria question, T indicates tota^l. 



"Absolute" refers to one respondent who .^i^ated al.l the 
problems as a whole,* without using ^specific cr^iteria. 



■ . 31 . 

TEST STRATEQIES . • * . 

"The test strategies for this experiment, were deveJtoped, 
using a -national survey of.deaign practitioners, to obtain 
a imiform measure c3f current opinion, which, was then used as 
a framework for strategy formulation by the experimenter. 

' A> -National S.ilrvey Questionnaire j • ' 

The national survey was based on a questionnaire mailed 
to drrdividuals prominent in the design methods literature of 

/envirpnmental , design and relateti fields. 'The purpose of the 
(^ttestionnaxre was to present alternative approaches to design 
strategies in a standard format iri order to obtain the most 
Current viewpoints stated in a' readily comparable manner • 

o 

Materials -for the survey were generated as: a)- "steps", 
listing alternative operations which. might be included in a 
design strategy; and b) "assumptions", describing alterna- 
tive approaches which might be used in formulating a specific 
^et of -steps in a design strategy. Thevsteps were based on a 
synthesis of a number ,of design strategies (Appendix B^l) 
with an emph^si^^ on steps which seemed to have^^^a^ 
acceptance, bu!t which indicatecj basic differences in meaning. 
A total of 32 stegs (Appendix B-2) were identified in tha . 
following general categories : • 
a. Definition • . e. Solution Development 

b-. Planning : ' ' ' ^ < f , Evaluation • 

c. " Information and Organization * g. Implementation and Follow- 

" , • " " Through . ^ • • 

d. Solution Generation ~ 
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In a similar way, the assumptions* \<?ere';Srenerated from the 

* ■ /• • - 

design methods literature;- although in this instance ' the con- 
cepts^ were usually not stated expllci^tly ^and had to be extrap- 
olated on the basis of overall ^ppr^oaches to design strategies. 
In the final outcome, 19 basic assumptions (Appendix B-3) were 
generated in ^he following categpries: , # 

a. Strategy Function c. .'Specific Operational Approaches 

b. Design Process ' d. Overall Problem-Solving Approache 

The survey requested respondents to indicate the sequence 

of steps for an effective design strategy, and to signify 

their' agreement or disagreement with the assumptions which , ■ 

were "of f ered. - In bath instances the order of presentation./ 

was varied from the groupings indicated above, in an effort 

to t>reak up obvi^ous sets and encourage independent judgments. 

Forty firospective respondents were identified on the basis of 

^ * ■ « 

demonstrated interest in design strategies^ and questionnaires 

were mailed to these individuals along with ^ covering letter 

of * explanation and a pre-addressed and stamped return evelope. 

B, Survey Results > 

A total of 13 respondents , representing important methods 
areas, responded to tli^ questionnaire . The results are shown 
on Tables 3-3 and 3-4y^ and summarized in 'Appendix B-4. The . / 
relevance of the questionijaire was essentially confirmed, 
since each step" and?, assumption was cited at least once, and 
most were mentioned four or ^more times. Typica]. step results. ' 
mcl'udpd a) support for th-d^- commonly recognized sequence of 
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"definition"/ "information gathering and organization'*/ 
"solution generation", "solution development" and "evalu^- 
tion"? h) twenty-four of the steps cited by a majority of 
the respondents, ^although with no consistent pattern or * 
sjquerlce; and c) an average strategy length of 18 steps, 
although in some cases steps were combined to approach a 10- 
step strategy. .Most .importantly, minority groups of respon- ■ 
dents selected specific steps not generally accepted, and 
suggested sequences of step usage different from the general 
pattern . . • , . 

The assumption results confirmed most of the assumptions, 
with at least minoi: acceptance for assumptions considered to 
be more "systematic" . An important aspect was the observa- 
tion that "th^ respondents frequently ' did • not discriminate 
between assumptions that were intended .by, the experimenter 

s 

* to be conflicting or contradictory. For example, the "Stra- 
tegy function'" assumptions were intended to represent mutually 

^ * . 

exclusive approaches, yet some respondents agreed with two of 
more of* these assumptions. Although, the responses ^ay have 
been due, in part, to the phrasing of ^ the asstiraption state-^ 
ments, ^it appears that thejre is a lack of a elear understanding 
or approach on the part of practitioners, which might be 
aided by an appropriate display and comparison of these con- 
cepts in the test strategies. . " 



Table 3-3 Step Responses on National Survey Questionnaire 
(numbers represent order of inclusion in strategy sequence) . 



ST^r ALI'LIU.ATIVLS 
Definition , Plaiininrr Information 
. (1) (2) C^O (5) (CO (?) CO (o)(ic)(ii)(i2).(-T^)(n^O(i5)(ic). 


A 




1 


■'2 


2 


2 
















1 












1 


? 




a 


16 






q 




11- 


12 


18 


n 

t 


1? 




c 


































D 


6 




■ 1 










c 


' : 

a 

7 


1 — 

lU 














E 




.1 




- ^ 


P, 




11 




6 1 


7 


10 


^^^^ 


1-4- 


.17 


'-16 


1? ■ 


F 


2 


1 






Zl 


6 






12 




14 












G_ 

H. 














J 










0 






■ 




2 


2 


1 


.1 


















♦ 








1 


1 . 








2 
























J 


























✓ 


' 2. 






K 


. 2 








,6 


/I. 














7 






t 


L 


6 


1 




2 


8 




■4- 


• 7 


10 


.11 


12 




9. 








:i 


2 






2' 


2 


"2 


■ 2 










•3 






*2 






10 


7 


7 


9 


1.6 


8 


8 


- 4- 


7 


8 


. 7 


7 


6 


5 


5 


2 



Generation- ■ Development . Evaluaticn Imp.-. 
(17)Cl8)(19)(20)i:21)(22)-(25)(2^r.)(25)(26)(27)C2S)(29)(30)(i,i)(32) 


A. 


4 














.6 












5 ,'6 


6 


6 


B 




2 


8 


'19 


20 


21 






19 






14 


22 


17 


25 


2?, 


C 










i 
























D. 












1? 




18' 


14-. 










16 


17 


19 


E 






20 




2? 


21 


2't- 


?9 


18 






19^ 


27 


26 


28 


?0 


F 




8 


■'9 


10 






18 




7 




17 




16 






19* 


G 


































H 




? 




6- 


6 


6 


6 


9- 


6 


6,7 




• Q 


7 


q 


9 




r 














- 7 


10 


• 5 


6 


•9 








8' 


. 11 


J 








■6 


















9 


7 


8 


10 




























1 








L 




IJ 


16 


. 17 


19 


21 




211 


'18 






14 


22 


20 


2? 


25 


M 




■ ^'^ 


. 8 


ij 








8-1; 




4-7 


10 


2 


1? 




7 




T 


1 


.7 


6 


. e 


6 


■■' 5 


7 


' ,8 


8 


_ 5 


4 


6' 


8 


3 


9 


"9- 



A-M indicates respondents'; C,G omitted this section; Tf total 
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Table 3-4 As sumptio-h Responses on National 'survey Questionnaire' 



'x" represents agreement with assumption ; ; ^ 



ASSUMPTION'' ALTE'RNATIVES V ; 




Fun 


Gtion . Process v ODerational ' Owor-^n 




(>a) 


(b) 


(r) 












(1) 


(d) 


(k) 


(1) 


(m) 


(n) 


(o) 


(P) 


(q) 


(r) 


Is) 


A 




X 




, 1 








J\. 














X 




X 






b; ■ 




X 




















X 




X 


• • 


; — 






X 


c 


X 


X 


X 






, A, 




X 




X 


X 




X 








X 


X 




D" 












X 




X 


X 












X ' 




, X 


• 




E 


X 


X 




X. 


X 


















X 






i— 






F 


X 








X 












X 


X 


X 




X 


X 








G 


± 


X 












X 






< 






•X 


X 




X 






H 


X 


X 


X 


X 


X 


X 




X 


X 


' \ 






X 




X 


X 


X 






I , 


X 


X 




X 


X 


X' 






X 


X 


X 








X 






X 




J 






X 


X ' 




X 


. X- > 


X 


X 




X 






X 




X 




,x 


X 


K 




X 




X 








X 


X ■ 


X 






X 








X 






L 






X 






X 


X 




X 




( — : 

X 




X 




X 




X 


X 


• X 


M 






X 




X 


' X' 


'x 












X 




X 








T- 


6 


8 

• 


6 


'5 ' 


6 


8 


2 ' 


9 


6 


4 


6 


3 


5 


5 


8 


4 


9 


•5 


3 



NOTEi 



A - M indicates respondents, T indicates total^ for all 
respondents 



C. Strategy Development . ' 

: ' The development of the specif-ic strategies was based on 
the experimenter " interpretation of the survey 'results, modi- 
fied |)y, the requirements of the research study. The maoor 
^ experimental consideration was the "comparability" of strate- 
gies, requiring the provision of distinct variations which 
would be likely to produce measureable effects on design,, ' 
performance. ^The degree of guidance in each, strategy was 



initially varied by specifying* three .different' lengths or num- 
bers -of steps in the strategy se'qijerice. This approach is 
suggested by the stirategies of various lengths as shown in 
Appendix B-1. Lengths of 5», 10, and 15 steps, were selected 



£n>Q£i3fer, to have comparable 'numeric* differences, while main'- " 

taining the '-^s trategies within manageable lengths . 

• .■. ,' ♦* 

; .(V.- 

• * : • ' . ' .* ' . 

The structure and content of the strategies were also 

varied in ways which might highlight the differences,^ P^^i" * ' 

marily. through developing the strategies in increasi^ng orders 

■ •■ _ • ■ . . 

ofci^omplexi ty (Appendix B-5). . The S-step strategy presented"' 

the simplest, most |Dasic approaches, eSsent^ially redefining 

the m^ajor- divisions in the design process, apd forcing reliance 

on, individual intuitive approaches. The lO-step strategy was 

based on a more traditional modeL using the general sequence, . 

and most popular steps arid assumptions . identified in the 

• . ' \ ^ \ . . \ ; ; ^ - 

survey. The lev^el of detail provided was increased over the 
5-step. strategy but the^ 10-step alternative foJ^lowed the giqst 
obvious patterns, and required considerable individual 
interpretation.. "The 15-step strategy was - developed by the 
experimenter "as a theoretically "best" design" strategy , 
utiiizir^.^^resumeafily^^^ systematic concepts pre-tested"" 

and cohf irmeii^ by " at ^leas-t a minority of respondents in the 
survey. This strategy iricluded operational instructions for: 
initiatiri'g the problem-solvdrig activity;* controlling informa- 
tion releAj^ahce; and. v/dltition deyel.opment;, jproviding specific 



design aids. to' guide the subject throughout the design process, 
The. three strategies (Figure 3-1) were then assumed to include 
differences in -clength, detail,' and technique which would dir- 
ectly affect the' quality of (design performance. 



Figure 3-1 Test Strategy Step Sequence 
5- STEP STRATEGY 

1. Defining The 'Problem 

2. Gathering Data j 



3 •Developing Preliminary 
So.lutioji 

4. Detailing The 
Solution » ' ' 

5. Evaluating Solution 



10~STEP STRA^^EGY 

1* Determining , User 
' Requi^rements 

2C Determining Solution 
Constraint^ J ^ 

3. Gathering pata ^ 

4. Organizing Data ^ 

5. Developing Conceptual 
Solutions 

6. Developing Subsystem 
Solutions 

.7. Developing Overall 
Solution 

* ,> ' 

8. Detailing Overall . 
Solution ' - 

9. Evaluating Solution 
10. Revising' Solution* 



15-STf]R STRATEGY . * 
1. Identifying Design Task 



2. Deteirmining Operational 
Definition 

' /« 

3. D*etermining Minimum 
Limitations o 

4. Developing Solution 
Targets ' 

5. Screening Solution 
Targets • • 

6.. Gathering D.ata . . 

7 . Procesging-Data ^ 

, 8. Identifying Subproblems 

9. Developing^ Stitjproblem 

Solutti'dns 

10 . Integrating .Subproblem 
Solutions "\ 

11. Developing Splution 
System Alternatives ^ 

12. Detailing Solution 
Alternatives - . 

13. Evaluating Solution 
^l£ernatives ' . ^ 

1.4 .^Optimizing Solution 

15. Refining Solution 
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In the final form of the strategies an effort was, made 
to preserve strategy differences while minimizing extraneous 
factors as follows: a) strategies were not intended to 
''trick" the subjects, each of the strategies was workable and 
.present^ed as straight- forwardly as possible; b) roman numeral 
were used to identify strategy steps in order to highlight the 
important functiori's of each single step;' c) ' step titles were 
kept as uniform as possible between strategies, except for 
s'^pecific differences in meaning; and d) both title's ^nd 
explanatory definitions were reviewed for clarity and^ambi:^ 
guity • " ' . 

D, Strategy HypotheseSo . 

Spe<:ific hypotheses to be tested in the experiment were 

deyelpped.ini terms of assumed differences- in the strategies 

(Appendix B-6) , the general Hypothesis and subhypotheses are 

listed as follows : . ' " . ' , • 

General Hypothesis : Different strategies will produce signi- 
ficant effects on design problem-solving • , ' 

•. . ' ■ ■ 

Hyp, 1 : The 5-step strategy should g^enerally be more ineffi- 
cient/ producing unsatisfactory isblutions/ 

Hyp» 1-1 : The 5-step strategy should require more time for ' 
solution • V 

Hyp* ' 1-2 : The S^step strategy should require grea^ter ' effort 
getting started on prbblem-splving, 

Hyp> ^ 1-3. : The* 5-step strategy should be highly redundant with 
information io^iscriminately selected and used. 

Hyp, 1-4 : The 5-step strategy should show significant impcove 
ment in ef f iciency/ pvesp repeated te^ts. sessions^ 

Hyp, 1-5 : The 5-step ' strategy should produce significantly 
poor solutions, improving over time. 
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Hyp , 2 ; The IQ-step strategy * should be relatively easy to 
apply/ but should be generally oonservative , and somewhat 
inefficient/ ^ ' 

Hyp, 2-1 : The 10-step strategy should facilitate the attart of ^ 
problem- solving with litjtle hesi^tion, , , 

Hyp , 2- 2 ; The 10-step strategy should display considerable 
redundancy and inefficiency, wirth*a relatively, great con- 
centration on details, , . 

Hyp ,y^-3 : The ■ 10-step s_trategy should rely oh practical 
considerations racher than conceptual approaches. 

Hyp, 2-4 ; The 10-step . strategy should show relatively ^ 
slight but constant improvement ■ in efficiency over tjiree 
test sessions, ^ > " 

Hyp , 2- 5 ; J The 10-step strategy should produce moderate/ 
traditional, solutions with little improvement, over time, 

J. ' ' ' • ■■ ■ ■■' , 

Hyp , 3 : The 15-st6p strategy should be overall the mos't ^ 
efficient and effective, _ 

Hyp, 3-1 : The 15-step strategy should provide a direct, 

ef fective definition -of the problem, « ■ 

Hyp, 3-2 : The 15-step strategy* should utilize information 
selectively, with aflj^e f'fici en t .problem development. 

Hyp, 3-3 : The 15-step stra-tegy should concentrate on , con- 
ceptual information, developing broad solution implications, 

Hyp> 3-4 : The 15-step strategy should be relatively efficient 
•dnitiailyy with little change-over time, ■ ' _ 

Hyp, ^3-5 : The 15-step strategy should produce consistently . 
'high-level solutions with an emphasis on thoroughness and 
innovation. ' . _ ^ 

_ INFORMATION BANKS . 
The ■ development of -the three information banks involved: 
identifying ■ tKe kinds of information needed, generating informa- 
tion items Srom the literature, and structuring the information 
for the subject ' s use . ' 



A.. Types- of Information 



Major types of information, applicable to the three 

problems,' were identified from the test strategies which, 

either implicitly or explicitly , direct the subjects toward 

-Che" ^acquisition of different types of ^ information and the 

expression of diffferent concepts. Information' needs were 

^ . ■■■■ ' • ■ ■ 

determined in the areas df : a) defining the problem; .bj 

developing the problem- and soTutidn requirements-; c) speci- 

fying 'J:he final solution. Specific " def ijiition " information- 

mentioned in the . strategies 'includes': .1) user requirements, 

2) solutijDn constraints , 3) external constraints, 4) design 

task descriptors, 5) operational -definition statements; 

covering both tietaile<3 and minimum "limitations . P'r6blem 

" develdpniient " > information is not-specified in the strategies, 

. ^ _____ 

except ^fpr the ''direction to "gather " infpirmatiori, but alterna^ 

tives conut|pnly identified in the literature (Appendix J) ^ 

. >i ■ _ _ ■ ■ ■ - 

include: 1) user characteristics, 2)~ user activities. 



3) design factors or principled, 4)'. man-environment research 
information^^j^nd* 5) background information on similar, problems 
" Solution specification " information is identified in the 
strategies at- various levels including: 1) conceptual solu- 
tions or solution targets, 2) subsystem or subproblem solu- 
tions, 3) solution elements to be included' in oveira 11* so lu- 

\ 

tions, and 4) solution spefcificatione or element details in 

• ' . • , ^ ^, , 

final solutions. EvaJ.uatio.n information would not require a 
separate information type since it would refer back to earlier 
criteria. P roces>s information would also be needed, to indicate 
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specific techniques or tyjies of relationships\ used in pro- 
. cessing the information. 

% 

B\ InforiP.dtlon Selection 

Specific information for the three problems \ was identi- 
fied from a variety of sources / most of which. .would be avaij- 
--able to a practicing designer. The different source types 
are identified in Appendix C-1 and the related literature is 
cited in the information bank bibliography (Appendix J) . How- 
ever, some source examples are: a) articles, texts A ^d 
handbooks, related to environmental design; b) design currxcja- 
lum materials; c) plannin,g materials used for designing edu- \ 
cational facilities. The search for information was as "dompre- 
^ hensive as possible. Additional details might have be^n lo- 
cated, but in most cases these would have been beyond the 
reach of the typical designer. 

The selections of information from^ these sources were , 
controlled by the experimenter. The experimenter essentially^ 

, acted as a design practitioner preparing information for his 
own project, but including an expanded range of information 

^ alternatives to allow fpr different subject approaches. 



Selection procedures included: a) literally recording.-4nf or- 
mation as given; b) -screening information for applicability 
or reinterpreting it in a more^ useable form; c) fabricating 
information unique^ to the ^t^eriment '(as in the "client" 
requirements) , or . unavailble in the literature (new items .for 
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"vending"). This fabrication,, however," was kept to^a minimum 
* * ■ . " . ■• .. • ■■ ' 

and was always. based on general design principles. It shoUld' 

be noted that {\rith the exception of clearly outrageous state- 
ments, €he information was not checked for.aScuracy or rele- 
vance, since it was assumed that in the real-world the designer 
is often presented with u^icertain information and it is one 
of the functions \^ the strategy to aid in dealing with, s.uch ' 
information. ' ^ * . . 

C. \-^^^t!^m Generation . , ^ ' ^ ' 

• < 

The assembled information .was subdivided to formi speci- ^ 

* . . . . „ 

fic information "items" as shown in Figure 3-2. This involved 

generat:ing:' a) lists of single items; 6) modified it^ms 'or-, 
brief statements; c) .. longer statements- representing defini- 
tions or concepts; and. d) drawings of solution elements or ^ 
typical' layouts , Considerations^ in generating the items 
included: 1) logical and consistent subdivision of larger. 

concepts;. 2) familiarity and .understandability of terminology. 
» ' J. >w . 

and definitions; 3) ease of combining items* into complex / 
•expressions; and 4) control of item quantity in each type' ^ 
■category to provide a range of alternatives (usually 10-20r) 
without overloading the subjects^ - ■ ^ 

D. Information. Structure * ^ * . ' 

The collection of individual items was "organijzed in 
order to: a) facilitate subject location and use of items; 
and b) to simplify the coding and inter|:)retation of items. 



Each item was given a sp'e2cif ic location in a k'iera-rchical »^ 
system including: ^ 1) major information ^ fun'ction^- (using an 
alphabe-ci^c character.) ; 2) specific inforrri^atlo^ characterise/ 
tics (usually by "^tens" digit); and 3)' individual item" , \ • 
sequence (usually "unit" digit) . T?lis informational stru<fture 
was governed by: a) 'rough sequence of use,, to fcTcilitate 
location and retrieval; ^b)^ randomization of specifi^c use 
sequences, to minimize probability of - a straight §eq|uence of^- 
selections-; c) randomization of information i£ems v\jithiri ^ 
categories, to avoid experimenter bias. ;Ln determining item 
importance. Terminology in organizational- headings »was' chosen 
to maximize identification and recognition; and. minimize simi- 
larities to terminology^ used in defining test strategies. ^ 
Category descriptions are given in Appendix C-2. The main' 

headings used for all three information 'banks ' are shown as " 

\ . ^ . ^0 . ■ , • ■ ' ' ^ ' - 

follows : , • * . 

c 

definition Information 



A. Problem Descriptors 

B. Process Descriptors 
C' Overall Requirements 

D. Information Requirements 

E. General Requirements / 

Development Information 

F. Us^r Types ■ 

G. Action Types „ ' 

H. ^Design Considerations 
J. Research Information- ^. 



K. General Infoifmation 

* 

^ Solution Description ; 
7j. General Element Classes 
Z Solution Concepts ' 
W . Graphic Elements 

^X~*Y,« Element Specifications 

Solution Elements, 

M-N. Material Elements 

0-Q. Furniture Elemei;its 

R"S. Equipment Elements 

/ ■ I- 
T-^V. Ancillary Elements 



•TEST .APPARATUS, " . • * 

The development of the apparatus represented S key 

:i ^' ■■ • ^ . ■ ^ 

r> el|fement in^ facilitating the 'practical applicat;ion- of the 

' ! ^ / \ ^ ■ . 

/• • experimental materials in a real-life-like setting. The 

' ^'. . ! •■ ' ' 

"^pi^opedure used was to identify, a set of working requirements 

• I / . ■ • ' , ^ " ^ ■ 

based on the research needs, and then to determine the physi- 

: ' ■ . 

cal components which would best meet these requirements. 

' i ' ' . " - • 

■ Ai> Ope r actional Requirements 

The major consideration in apparatus design was to 
permit the subjects to work as naturally a^ possible, per- 
forming a number of different operations as described in 
, , Appendix C~3), and listed by major headings below :3 - >^ 
1\ Information SelecJtion • 

2 , Information Grouping 

3. In^rmation Relationship ' a 
' ' 4, Information Disposition 

5,' Information Modification ^ 

' * . * ■ I 

' . 6, Information Consideration . 
It was considered important to permit all" subject operations 
to occur^with a minimum of Effort, and with a natural flow 
' ' of visual—and physical movements.' ' ' : ' ' 

Re'co.rding Requirements - - , ^ . • * 

The apparatti's also had to accommodate the recording of 

' ? / specific data re-quired for the • information processing proto- 

. cols, encompassing the following dimensions described irf detail * 
* . . f ■ '■ ■ 

in -Appendix C-4: • 

•'■ ' ■ : ■ -SO, ■. 
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, A. Identification , ^ ^ ^ , 

B- Sequence ' • 
^ C. Dui-ation 

' ■ ^ ^ ^ ' ^ 

Description . . , it 

E. Step Behavior . . - 

It was important that all of these recording operations ^ should 

be accomplished as unobtrusively ,as possible, minimizing sub- 

^ , . ■ • V ^ ^ ' - " 

ject distractions and secpn^ry task burdens. 



C, Practical Requirements • 

• Additional ^requirements were considered for the best use 

V 

of the experimenter * s efforts and .resources , including: 
a) the use of three sets of apparatus, permitting simultaneous 
running of , three subjects to reduce the ampun'tof time required 
for testing,- b) economy of constructing and transporting 
multiple' sets of apparatus^ ' c) durability of apparatus to 
withstand repeated useage and handdJfng; and d)'^»i»pinimal de- 
pendence on experimenter for apparatus operation. In longer 
terms, the record produced by the apparatus, should be cpn- 
venient to, handle and process i-n coding and analysis. 

D. Design Process Information Recording . System (DEPIRS) 

DEPIRS (Figure 3-3) r^jSresents an integrated set of 
components adapted to' design prt^lem-solving research, which 
should be ideally suited for .this study and readily applicable^ 
to future projects. The apparatus development is shown in 
Appendix D-1, with the detailed specifications in D-2. The 
basic approach uses physical information cards which are * 
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selected and manipulated by subjects, wtth a photographic ) 

record taken automatically at pre-de termined intervals, pro- 
■ * . •» ' 

^ vida^ng" (3ala on information us^ The major elements of the'^^'ap.- 

paratus include: a) a IGxm motion picture camera, with b). an 

a^ttached timing mechanism, operating a single-frame exposure 

release every 30 seconds, recording the manipulation of c) in- 

. formation items pre-printed on-' light cardboard backed by a - 

small magnet, whibh allows ';them to adhere to d) a 3 ' x 5 ' 

metal chalkboard, resting oh eV'^ removeable base;, after the 

cards are removed from f) a three-ring storage binder with ' 

metal pages- Standard chalk and erasers could be used with 

the items to perf(^rm necessary modifications 'clarifications . 

-Figure 3-3 DEPIRS Apparatus . . ' 



DEPTRB was design^ for use in a classroom at the School 

> * • - . 

of Architecture, UW-Milwaukee' (although it could be easily 

adapted for other locations) which was particularly suited foj- 

' . the 6xperiinent (Figure 3-4> with-: ""a) movable display panels ' 
... \ . . . . ' 

, which could be arranged td^i form test ^'booths"; b) adjustable 

spotlights to illuminate -the display ar-ea; c) work table's to 

sipport the apparatus ; d> chairs for ^eating; ande) unused 

alcoves, for storing larger apparatus ej.ements between, ses^^ions. 

The ^apparatus , was set up as shown, following the procedures 

described in Appendix D-3. . , V 

Figure 3-4 Test' Room Layout . ^ , . 




Scale: a/8"= 1 
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TEST SUBJECTS • 

•■ ^ : . ' , . . ' . . ' ' • 

Subject selection based on a determinatio# of the 

characteristics desired in a subject population, identifica- 
tion of a ^suitable population, and^ selection of /:th« best' 

■"■ ' ' ■ ^ • ■ 

possible set of subjects. \ ' f^^ ■ 

A. Subject .Reqjlreinents ' 

\ ^ 

Subjects were previously identified as students trained* 
at ''an advanced level in ^environmental design problem-solving. 
Additional requirements included: ,a) membership 'in a size- 
able pool from which a uniform subset ^could^be drawn; b) ^ parti 
cipation in a common standard curriculum; c) some exposure to 
formal design methods, without being^ trained in any single 
stifategy or method; d) high motivation to • cooperate with a 
somewhat unique and tedious design task; and e) ability to 
spend time on three all-day test sessions in additl^on.to 
orientation, and. study time. ^ x 

B, gubject Selection * ' 

The nearest available students with suitable character-. 

• . ' ■ . • * * ■ 1 

istics were graduate students at the School of Afrchitecture,^ 

University of Wisconsin, Milwaukee (the department on the 

Madison campus has few st;,udents and their backgrounds are very 

diverse). The students in Milwaukee were reviewed with the aid 

of a cooperating facility member to identify individuals who 

had completed course work related to the scale of th^e proposed 

test problems,, and who seemed suitable in terms of level of 



comj^etance -pand potential inteirest. Tha records of the candid- " 

dates were reviev/ed to insure a similarity in: a) undergradu- 

ate . education, b)' gf aduatevprogram, c) work experience/ d) 

academic performance. Using this information it was possible 

to i^dentify a matched -group of ''nine subjects with several al- 

ternates^. , The "nine students were, further divided into t^ree 

equivalent grpups on the basis of .the' faculty member's class 
' ■ ., . ' • . * ^ 

•experience' with the specific ijadividuals . ' • 

. ' * I* 

" "C. ' Sub:]ect Recruitment 

The motivation of subjects became a major factor in the 
^ final recruitment of prospe^ctive test subjects. Initially, 
the experimenter, aware of the effort involved in the experi- 
ment, had specified a stipend of $9,0 for each soibject. This 

' o.' ^ . 

seeme<3 to be a substantial inducement, averaging $3/hour for 

f- , ■ . . ' . . , • 

three and a half day's work.. However, when the students 

were informed of thie nature pf the experiment and the stipend, 

only six ^of the nine-, candidates responded. The experimenter 

then determined that many of the students hf^ld outside jo'bs 

with an adeguate income and therefore the stipend was^ not 

sufficient to induce students to give up time near' the' end of ^ 

the semester.. Coiisequently , the stipend was raised to $150, 

■■ ■ ■ . s 

and on this basis the recruitment was' satisfactorily completed, 



CHA'PTER IV - IMPLEMENTATION: Operating Procedures 

Introducti*on: * . ^ ■■ ' 

The procedures used by subjects in the experiment welre 
left up to the individual subject's own working patterns. In 
contrast', specific procedures Were required in attempting to 
insure a thorough orientation of sub jects before the experi- 
ment, and in generating meaningful results from the data re~ 
cord after the experiment. . Additional procedures demonstrate 
the experimenter ' s responsibilitieis in handling the apparatus 
.and the method for running, the experiment. «^ 

SUBJECT ORIENTATION 

The oir^ientation of' sukijects had two main purposes: 
a) presentation and interpretation of -the strategies, to ^in- 
sure that 'subjects would be able to use a given strategy in 
practice; and b) familiarization of subjects with the tasks 
to be performed, in order to reduce accommodation time in 
the test session. ^ v ^ 

' \' ' ' ■ . ^ 

A. Invitation 

A memorandum waa, used to inform potential subjects of 
their selection for. the ^tudy, and to briefly describe the 
nature of the task and tbe time committment required. Sub- 
jects were requested to •<:ontact the liason faculty member 
to register their acceptance and to request any needed clari- 
fication. - ^ • . * ' . . 



■x^ ' -.52 

. ■ ' ■ ^ ■ 

B. Oruajitation Meeting, , . ■ * 

A meeting, was held with the experimenter,, the liason 
faculty ffiember, and six of the subjects to answer questions 
about t]ye experiment and worJc on scheduling (additional sub-' 
jects were handled individually) . ' The experimenter expanded 
on the information provided in the invitation, avoiding the 
specific discussion of * problems^ or strategies. The bulk of 
the meeting was concerr^ed with makrng^r decisions; to;/ a) run , 
the test s-etssions on^ three consecutive weekend/; b) work on 
Saturdays,' Sundays, and mondays; c) start th4 sessions at 
9:00 a.m.; and d) schedule the .weekends within the time re- 
maining i^ the school term. . ^/ 

C. Or.ientation Booklet ' 

An orientation booklet was prepared for distribution 
* 

to -subjects one v/eek before the first test session. This 
booklet provided instructions for .the experimental task .fo^ us 
as a reference throughout the experiment. fhe contents of the 
booklet are listed b-elow, they essentially pre3^ented metterials 
covered elsewhere in this repor-t: . ' ^ 

1. Procedure Summary . - , 

2. ' Experiment Oveirview . " 

' 3. Design. Strategy Introductioh . > 



' .4. 'Design Strategy Description ^ 

5. I*nformation Bank IntrdducticSn , 

6. ^ In'f ormation Bank Structure 
v 7. Experimental Apparatus 



8. Using Information Items 

9. 6sing Notations (chalkboard) 

10. Pretest . , * 

11 Tentative . Schedule of Test. Sessions 

, A major function of the booklet was the distribution of 

specif ics test strategies to their respective, users, 'listing 

the specific information for each str,ategy under "design 

*A ■ . . • ■ ■ ■ , ' . — 

strategy description", while holding the rest of the booklet 

constant . The l^argest amount of space in the booklet was 
■yused for orienting sub jects to ^specific functional aspects 
of the experiment. An attempt was made to describe the de- 
tails as fully and operationally ^s. possible, without making 
reference to the problem types, in an effort to prevent ad- 
vance planning. or- thinking about a specific method of solu-- 
tLon. 

The pretest was des"igned to serve; as a study aid and to 
be used in the pre-sessipn orientation as a check on sxabjeict 
comprehension. The test used "matching" and "fill-in- ther^ 
blank" questions to verify subject understanding of ^ the de- 
sign strategy and its applications, and the structure and 
use of the information banks. Some questions were left. open- 
ended' in order to elicit general questions an(J uncertainties 
about the materials . 

D. -Pre-Session Orientation * ' . 

- The exp"^erimenter met with the three subjects who would 
be using the same test, strategy on any given i^est day, at 



the start of. the* test session, taking aboT4t one hour on the 
initial sessions to cover: a) review of the pretest and dis 
.cuss ion of the., test strategy, with exajniple's and reference to 
questions raised on., the pretes.t ; b) ' a demonstration of the 
apparatus^ and its use , literally paging through the anforiiia- 
ti^n ^bankP and reviewing the information contained in eacj^h ' 
secti'on; and c) a brief statement of the. problem^and a dis- 
cussion of its implications, stressing an innovative and 
creative- approach. 

Pre-session orientations were given on all three week- 
ends with the subsequent sessions taking less^^time to cover' 
the prpblem statement, and new information. In all the ses— 
sions the subjects were encouraged to ask' questions, and ori- 
entation was continued until siibjects appeared confident in 
comprehending their task.' 'An effort was made to insure uni- 
formity between, orientations for different' groups , using -a 
standard format\. and repeating similar information . in the 

same way . ' ' . . ] " 

/' • ' '* ■ ■ ■ . . ^ 

EJCPERIMENT PREPARATION • 

The preparation '^or the experiment consisted of working 
with the apparatus so that it would be fully functional at 
the -^beginning of the test sessions'! _ . N *^ 

A.. Pilot Test ^ ' 

• - . V , ■ 

After fabrication was completed, the apparatus was 

' ' ■ ■ ' 64 X/ ■ ' " 
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a^sseinbled for pretesting^ The set-up was revlpwed to check 
for completeness ^nd convenience of opera tidn. A graduate 
student in Industrial Engineering was asked'^to review* the 
of-ientation* booklet/ and then to use the apparatus in an . • 

abbreviated tei^t session. The results of this evaluation. 

.■}■■' . . ■ ■ 

^ * ... ' . • 

generally confirmed the applicability of the orientation 

booklet and apparatus design. Some suggestions were made 

.for the prie-session o^ientations^ including: a) clearly 

explaining ^trie operation of the apparatus;; and b) stressing 

•that subjects were ^ot to work competitively. Technical sug- • 

.- .-■ ^ ■ ■ ■ ^ ■ -.H 

gestions included:' ^) writing descriptive information on j 
the chalkboards to identify each film record, 2) increasing j 

the camera distance to cover a larger work area, and 3) using! 

- ■ ■ ■ - ■ - ' 1 

a formai written log to r^,Gord subject questions and comments,* 

during the test sessions. . - Ci i 



B . Transportation 

The completed apparatus was transported to Milwaukee 
two days, before the first test session in a station wagon 
which readily accommodated: * the ^ three chalkboards and bases^ 
(in a .folded . position) ; three tripods; and four boxes con- j , 
taining cameras, information banks (in storage binders) andl 
apcessories. The experimenter and one assistant easily l0c(ded 
and unloaded' these materials and installed them in the tes|: 



site . 



C, ,p Set-Up 

The apparatus was set-\up in tl^e test area following jche 
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procedures -^detailed in Appendix D-3. TKe major aspects in- 
cltided: a) clearing and ^^eariranging the test^^^roora to accom- 
po<fate the apparatus, b) . placing the larger apparatus items 
in position,, c) unpacking . and distributing the materials in 
tKe work areas. This procedure -las ted only a few hours, with 
the. heavy moving accomplished the previous evening, and the 
final details completed before the .pre-^session orientation. 
The interim** use of the roojna r'^i&q-uired this set;-up to be^rfe- * 



peai'ted on each of the test weekends.. ' 

RUNNINGvTHE EXPERIMENT ; > ^ * * * 

The procedures for runnilig th,e tesjb sessions we're*. fc^r/* / 
the most part '^open-en&ed, ' allowing subjec ts *to ^work Out prob- 
lems in theij: own way, Vith a few formal aspects used to 

maintain experimental control . The overall timing 'of the 

■ ' ' . ■ 1* 

experimental session is given in Figure 4-1.^ 

A. Subject O^Berations , * 

AM: the conclusion 6£ the pre-session orientation, each 

- «• .. r/ ' ^ - • ( '' . .' • 

subject took a^^ position at one of. 'the test stations and wrotd 
out identification ^information on the ""chalKboatd. When this 
was completed, the cameras and timers were started to begin, 
the recording process . ; Since* the timers * operated at fixed 
intervals, -no external . time-keeping was required. « • 

Once the cameras, were started, the subjects began work- 
ing, usually looking through the items in the information 
banks, starting with materials related to problem definition. 
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Table 4-1 .Times .Recorded for Experimental Sessions 
(figures ^irfdicate time at which event jOccurred in 'each session) 



EVENTS 




TEST SESSION'S ' \ 






\ Problem II , 


Woblem III 




10-st 


15-st 


MOM 

5-st 


I c Am 

10-st 


o QtTNT 
C UiN 

15-st 


MOM 

5-st 


10-st; 


'C TTM 

is-s-t 


Mom 

5-st 


Start 
Set-Up 


r 

r 

8:00 


8:00 


8:00 


_ 8:00 


■ 8-:00 


8:00 


8:00 


8:00 


8:00 


Start 
Orient . 


11: 00 


9:45 


9: 30 


9:00 


9: 15- 


■ 9:1Q 


9: 30 


9: 15 


9:15 


Start 
Testing > 


11: 50 


11: 15 


10: 30 


9 : 30 


9:45 


9: 40 


9:45 


9 : 30 


9: 30 


Lunch 

Break 

'— 


1:05 


12: 30 
f 


12: 30| 


12: 30 


12:40 


12: 40 


12: 35 


12:00 


12: 30 


ReSta-rt 
Testing 


1: 50 


1:05 


f:45 


. ^,= 15 


1:20 


1:40 


1: 35" 


12:50 


1:45'^ 


Winding 
Break 


4:00 


3:30 


3:15 


4:00 




5:25. 


2: 50 






Res td:rt 
Testing>r 


4: 15 
« 


3: 45 


3:45 


4:10 

< 




5: 35 

s 


2:55 






End 

Te's ting 


7:45 


7:00 


8: 00 


5:10 


5.: 20 


6 : 15 


3:25- 


4:30 

i 


•5:00 


End 

Session 


8: 45 


8:00 


9: 00 


6:10 


6: 20 


•7:15- 

. f • 


4'r2^ - 


5:30 


6:00 



"find Testing" .represents ' time at which last sub*ject stopped 
working on the pr^oblem 

"End Session" represents time at which all- cards were replaced 

in banks, subjects were dismissed, and experimenter completed 
_teardown . ^ 

Subjects were.^ f ree to work at their own pace and pattern, 

within their interpretation ..±h.e deslgii..,.s.t.rat^^ The only 

formal requirements, were that they should indicate the strate- 
gy step they were using at any tim^ placing an approt)riate . 
indicator card in the display area; and thai? on the first prob--- 
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lem they, should descriKfe any layout 'configuration using ^a *stan- 
d^ard io^p^tional cpordinate system • (a^attempt to standardize 
data coding) ... ' ' » • - 



Typical work patterns consiste'd of a cyclical sequence* 
of selecti>Qn, organization, and interrelation of information . 
i.tems; as information ^was used for: definition,* development , 
and solution (specification . Individual diff erences ' were pos- 
sible in termk of the quantity 4nd i:ype of . information selected 
different sequences and configurations, and different use of 
Vjsrbal and graphic notations (Figurfe 4-2).' " ; ^ 

■Figure 4-1 Typical Information Displays ' , ' - 

' a * . ^ * ■ » • V - ■» 

. - . (d) 



.(a) 
(b) 

(c) 
(f) 



TTTT 



n 



C30 



If 



TTTT 



1 



5ii 



ie) 



(g). 



(a) Straj^egy Step 

*(b) rt^m ^Selegtion 

(c) Item Srouping 

'(d) Relat>ionship . 

.(Series ^ General) 



(e) Disposition (pric^rity) 

(^f ) Verbal Notation * 

(g) Graphic^ Notation . 



B. Experimenter Operations ^ 

The experiment^' s major role was as a monitor, super- 
vising subjects and apparatus to guard against accidents or 
difficulties. The experimenter did not interact with the sub- 
'jects during' the experiment, except for a few brief, questions 
shown in Appendix E-1. The relatively complex camera set-up 
was checked frequently (Appendix D-3) to insure that the * 
record was being made satisfactorily. 

C. Break Periods . 



A luncheon break of approximately one hour was scheduled, 
at which time the cameras and timers were stopped. Subjects 
were cautioned to avoid either looking 'at each other's work 
or discus^sing the experiment ciuring break periods. Bokh this 
period and a similar mid-afternoon breal^: -were used to rewind 
the cameras. Afte^r each break,^all subjects reassumed their 
positions and the cameras were restarted. Informal, restroom, 
and "stretch breaks, were permitted for individuals, but the 
ccuneras remained in operation for these brief periods. 

D. End Session 
' — ^ ^ 

r 

The conclusion of work on the tes't problem was left to 
the discretion of- each individual subject. Usually the pat- 
. tern was: a) one' subject would announce that his work was 
completed, b)^ the, camera and timer for this siibject would • 
be Stopped, c) ° shortly thereafter the oth(sr subjects would 
conclude their. work (this could ^l^ake as much as an hour) , . 
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d) after a given sub:|^ect had completed his" work, he would be 
requested to return h'^k'^* cards to the information bank. 

E> Subsequent^ Sessions ^ ; ' 

A similar procedure was followed for^ alj.' nine test sessions 
over the three weekends, with some variaJ/lDns: a) design acti- 
vity started earlier on each successive weekend, 'b) the after- 
noon re-winding breaks were omitted in shorter sessions, and 
c) the coplrdinate ^location system was abandoned in favor of 
direct graphic layouts after the fiorst sessions. 

F. Debriefing Meeting • 

A meeting of the nine subjects was held shortly after the 

last test session, in order to obtain retrospective obs-erva- 

. tions on the experiment. The subjects were askec^ a series of 

questions which are given with their responses in Appendix E-2. 

The questions covered the following areas: 

/ * ' • ■ 

A. General Impressions F. Reaction to Informa- 
' tion Grouping 

B. Representation of Real ' . ^ 
Problen\-Solving Processes G, Reaction to Informa- 
tion Bank 

C. Ability to Follow 

Test Strategy • H. Different Ways to 

D. Diffei^?ice From 
Normal. Strategy . • ,1. Reaction to 



E. Learning From 
Experiment 



do Experiment 

Reaction to 
Apparatus and Method 



DATA -PROCESSING 

Once the experiment was completed, the film record gene'r- 
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• • 6 

It 

atod in each test session was displayed using a "single-frame 
16min projector. Data was retrifeved^ by means of a frameTby- 
frame inspection of the film record. The resulting data v/as 
used, to generate a, variety of ''performance" measures , using 
^oth machine and manual tabulation. 

A, Data Cording * ' . - * 

A coding procedurjs was used to translate observations of 
-subject operations on the film into a form convenient for 
further processing and evaluation. This coding procedure was 
developed as a means of effectively and efficiently represent 
ing the data in a form suitable for keypunching, ^The coding 
methods and " gonventions " are - described in Appendices F-1 and 
F~2, essential feature include : ^ 

1. Identification 7. Group Relationship 

2. Frame Number - 8, Relationship Type' 

3. Strategy Step. ^ 9, Disposition Type 

4. Information rtems 10. Verbal Type 

5. Information Status ' .11. Graphic Type j 

6. Information Grouping . 

Using the coding system it was possible to review and 
code the films relatively easily , noting only changes occur- 
ring between frames. The completed coding sheets were sub- 
rt- mitted to the keypunching service. A total of 17-^743 ca^ds 
Were punched and later transferred to magnetic tape. 
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B. £:omputer lobulation • . o 

A considerable amount of • data wa^ available in terms of: 

" ■ ■ ■ I . 

a) information use, h) groupings, c) relationships, a^nd 
d) modifications ; recorded by: 1) time, 2) sequence, and; 
3) strategy step for e^ch test session. For the purposes of- 

this report, within the, time and resource limitatibns, it was. 

... ' - - ^ ' 

only' feasible to use a' summary of this data: ^ -tabulating total 

scores fqr each measure over each entire test session. A pro- 

t • * - • 

• r 

gram was written to generate frequency counts and related " 
statistics for information item selection, reuse, and groupings 

Details are shown in Appendix G-1, ipajor informati on ^ types are. 

fi ■ 

listed as* follows: 

• 1. Item Selection (class and subclass totals) 
2. Item Reuse (total reuse, total reuse/item) 
.3.' It^m Groupings (total groups formed) 
4. Grouping RunfS (length of group sequences)' 



5. Tbtal Number of Cards (data) 
The program vras somewhat complicated due to 



the number of 



tabulation listings and statistical, functions, lj(ut the data 
dould be read by the computer very directly using the item code 
and group numbers. ' . ' 

C. Manual ,l^abulation . * • . 

Manual tabulation of additional data va^riables was em- 
ployed due-to^the relative expense of program writing. .This 
tabulation wa-s based on a print-out of the data cards in an 
easily r ea. dab forma t.j!Appendi G-2) . Using this method of 



hand counting .the following measures were obtained: 

1. Number of Frames/Session 

2. Number of Relationships Formed , 

3. Number of Disposition Occurrences 

4. Number of Verbal 'Notations . ' . 

■ ., 1 • ■ : . • 

r • . 

5. Number of Graphic Notatiohs ^ 

Although this procedure wasVtihe-consumingr the process 
•was simplified by noting only absoiutP^^ of occur- 

rences rath.er than ^specific types Wi±tki^ e/^ch category". . In 
addition, thesd variables were by their nature less numerous' 
than. the information item data' tabulated by computer. 

SOLUTION EVALIJATION 

'A separate procedure was used to generate measures of 
solution effectiveness, based on the ' solutions as presented 
in the film record, rather than the item-by-item boding. 
This process used direct inspection by the experimenter and a 
jury of expert evaluatprs to identify and order solution • dif- 
ferences : * " ' 

A> Solution Record 

Solution data occurred on the film record in terms of: 
a) graphic sketches made on the chalkboard, and b) specific 
in^rmation items -related to the solution,^ Th6 graphic images 
wdre- literally copied* from the film, using a projection and 
t:^facing technique, including no^tes and specifications written, 
with the drawings. Related solution information items were 



usually indicated by their placement ;or context and their code 



number^ were recorded in the groupings .shown by the subjects, 
fot later decodingv^ in written form. ^ ' . 

* ■ 

B. Solution Form at ^ ' 

/ ■ ■ . . , ■ • • • ' 

The. solution . data was reorganized for presentation to 

the evaluators and for general reference (Appendix. H-3, h4) : ^ 

a) the subject's drawings were rerdfafted using a standard 

format ito fit an 8^ x 11" sheet/ b) a perspective view was 

generated by the experimenter from the subject ' s drawings, 

c) the square footages of the proposed designs were e3timated,. 

and d) a solution specification sheet was generated/ listing 

the solution elements selected by subjects/ screened for dupli- 

cates/ and arranged with informative "element type" headings. 

C. Solution ^Evaiuators , ' 

A panel of Resign professionals was used to review the • 
solutions in order to establish meaningful/ expert compa]fc.s.ons . 
Individuals were selected on the basis of their professional 
standing to include a variety of orientations, toward problems 
of this type/ including: 

1. campus architect/ UW-Madison/ member AI A. ' 

2. Partner/ lot:al architectural firm/ with prior- experi- 
ence in the design of classroom buildings. 

3. Professor/ Department of Environmental Design/ with " 
experience in educational facilities, design/ AIA.^^'"' 

These individuals werd approached by- the experimenter 

who explained the nature of the evaluation tasjs and secured 

the cooperation 6f each person on the basis of their interest/ 



with^a small honorariun^ for the effort involved. • Evaluators 
were given three weeks to. complete their j udgments . 

. D- -Evaluation Criteria 

Evaluation criteria were generated by the experimenter 
to assist the professional evaluators in making their judg- 
ments, based considerations coiumonly applied £o design 
solutions, formulated in teirms.-of the materials availafc>le 
for evaluation (Appendix H-1) . Specific criteria titles were 
1. Comprehensive ^ 4. Aesthetic 

2/ Effective 5. Innovative 

3. Efficient . 6. Contplete ^ 

E.- Evaluation Package . . 

Materials were assembled and packaged in a looseleaf 
binder for- use by the evaJ.uators, providing explanatory, 
information^ in a form which would be convenient to handle 
and review. Major types of information included: 

1'. Description of the Experimental T:ask ^ , " 

2. General Instructions for Evaluation^ 

3.. Description of the Criteria • . ^ _ ' 

4. Listing of General Requirements 

5. Listing of Specific Problem Statements and Require- 
ments ' . 

•6. Nine. Solution Drawings for Each Problem 

7. Nine Specif ication Sheets for Each Problem 

8. Ranking and Rating Sheets for Each Problem 



. ■ . • . ' , 66 . 

The general and specific requirement statements (Ap~. 
pendix H~2) were developed directly from the ' information banks 
so. that; .evaluators would have available to them the same in- ' 
forma'tion given to s\ib jects . . The solution drawings and. speci- 
fication sheets were presented front and back in-, a plastic 
sleeve so that f:hey could be used together, and could be . - 
remo'ved from^the binder to facilitate ^eci*sioh-making. The 
evaluators were not informed of the use of different stra^, 
tegies and the order of presenting the nine solutions for each 
problem was^ randomized to eliminate strategy groupings and ' 
order effects.; Subject sol\itions were identif ied on ly by 
a code letter representing order of presentatj.oh. The evalu- 
ation forms provided for a two-step judgment/ ranking the solu 
tions^tb each problem, using pre-printed boxes.; then rating 
the solutions to show relative differences using the code 
letters written along a contiryious line to indicate ©gmpari- 
sons with "best" and "woi'st" expected solutions.. The results 
of t^his evaluation are indicated in the following section 
with the other experimental measures L 

F. Solution Characteristics * 

A separate maiiual procedure . was ^used to generate solu- 
■ • • . w ■ ' ' ■ , ' ■ 

tioh characteristic measures , - as an effort to reinforce the 

evaluations. These measures were tabulated from an inspec- 

tion of. the solution drawings and specification sheets. 

Details of the development are described in Apgendix I-l, 



characteristic types included: 

Niamber of Elements Listed in Specification Sheets 
2.. Number of Details Provided in Solution Drawing's 

3. Estimated Square Footage v , 

4. Number of Room Divisions 

' • . ^ . 

5. Number of Functional Roont A-reas ^ ' ' 

6. Number of Changes in Floor Level 
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CHAPTER V - INTERPRETATION: Results and Implications 

< * ■ ' . » ■ , ■• •* _ ■ 

Introduction: '\ ' 

The experimenta]^ result! were quite extensive' v^ith a 
large nxamber of individual measures of * sub ject. performance 

and solution' quality generate d fr om the experimental- data . 

. - ■ i? ' *■ 

However, the usefulness of the Results in determining the 
effects) of design strategies was limited by the- great, vari- 
ation in subject responses within strategy groups ^nd the 
lack of distinct strategy differences on. major measures such 

<» . 

as overall information useage, total time required for problen 
solving, and overall solution quality . * Despite -this, therfe ■ 
were a considerable number of suggestive differences in the 
response of strategy groups on specific measures which could 
be discussed in terms of different strategy characteristics. 
Based on these suggestive differences arid the overall config- 
uration of the results, modified by subjec.t f^dback in the • 
debrief ing session, . it is possible to develop som'e tenta*tive 
implications for future research in design <^ methods apd for ^ 
environmental design methods in general. ^ • . 



EXPERIMENTAL RESULTS ' * . * 

- The^ major emphasis of this experiment was on the devela^p- 

ment of pe4f:(^'rm^^ce' measures describing the sxabject.s' use of 

information while solving the test problems. Consequently, 

' . . .» • 

the major part of the results covers measures related to the 
overall patterns of information use, and the frequency of 
selection and reuse of information within specific informa,- 
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tion type categories. much smaller number of measupres were 
generated from -the inspection and evaluation of the solutions 
developed by the subjects. All of . the measures are- pescribed. 
in detail in the appendix and summari zed in the body/ of'the 
report. An additional type of result/ the responses given by 
subjects, in the debriefing^ session are listed in Appendix E--2". 



A.. 



Presentation of Results 



In the final results, seventy-one separate measures were 

. ' ' . . * ^ , • ' ■ ■ ' i 

generated for u^e in this report? The distribution of these 

measures, by categjory, is shown in Table 5~1, In/reading 

this table it should be observed* that category #li summarizes 

use* characteristics . over all types of inf ormatioi]!, while 

^ categoj^es #2 - #5 represent use scores for (different types 

of information. The speodfic measures witiiin^each category 

are described in Apper^ix 1-1/ . and individual sub ject scores 

( ^ : 

Table 5-1 Distribution of lyJeasures 

(figures represent number of measures, established 
in each category) 




TYPE OF MEASURE 


1 

\ 


# of' 

MEAS. 


1-. General Performance 




15 


2. Definition Information ' 




10 ■ 


3, Development Information 




10 


4. Solution Description Information I 




10 


5. Solution. Element Information J 




10 


6. Solution Characteristics | 




7" 


7. Solution Evaluation | 




9 


'ALL '1 
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for each test session are presented in Appendix 1-2, following 
the format described in Figure 5-1- For reference purposes 
the average scores for each strategy group over all three prob 
lems are. presente*d in Table 5-2. This table lists* all of- the 
measures with the exception of the overall solution ranking 
and rating scores which are averaged in the final e^try on. • 
the table. The measures in the^ table are identified by 
abbreviated descriptive -^itle', and specif ic important measures 
are further identified in ■ the discussion of specific measures. 
It should also be noted that the scores for each measure 
represent frequency- counts for the indicated dependent vari- 
able, except for the solution evaluation scores in category 
#7 which represent the. average of the evaluatorsi rankings 
and ratings, in which "1" was the best possible s^core. 

B.. Analysis of Variance 

. An . analysis of variance was applied to each of the" > 
measures^ in -order to determine a statistical measure of 
significance for 'different experimental variables. This 
analysis was used despite the acknowledged high variability, 
^in individual subject scores and the lack of statistical 
power in analyzing responses from a small group (^f subjects. 
The usefulness of the ''test was considered to be its ability 
to highlight special effects within the mass of data as a 
basis for further discussion. The anlysis was based on 
the model shown in Figure 5-1. Using this.model, levels of 
significance could be computed for the effects of: the test 

• . SO . -•■ • ' 



Table 5-2 Average Strategy Scores ' , i / 

(figures represent average strategy scor^, usually "frequency 
on a. given ineasure^ averaging three sub ject;s pver three prob- 
lems within a strategy group; see Si on Figure 5-i) ^ « . 



TYPE 


MEASUEE • 


STRATEGY 
5 I'O 15 


A 

6 

/ 

■f 

s 


T-tPE: 


. MEASURi: » 


STRATEGY- 
. 5 10 15 


1.1 . 

Gen- 
eral 
Per- 
forr 
.oiance 


1.1.1 # 

ems Select. 


jog 


256 


199 


0 

Def- ,f 
ini-' , 
tion^ 
Info- 
rma- 
tion 

ect- 
idn 


2.1.1 Prob- 
lem De scrip 


10 


•6 


■ 8 


1.1. 2# Card 
Generated^ 


634 


* 

768 


580 


2.1^2 Pro€-, 
e-ss De scrip 


1J 

^ 


^'3 


, 2 


1.1. 5/^ Fra- 
mes' Requrd 




6|8 


7^2 


2.1.3, Qyer- 
all Rjeq. . ' 


23 


15 


1-6 


? 

«J 

1.2 

Ka- 
jor 
Per- 
for- 
mance 


1.2.1 # It- 
ems Reused 


■■90 


115 


92 


2.1.4 Gen- 
eral Req . 


15 




10 


1.2.2 # Re- 
use' Occur. 


139 


215 


170 


ALL 

.(I 


63- 


35 


36 


A.^.^i^- Gr- 
oups Formed 


■118 


14? 


t 


3.1 ^ 

Dev- ' 

elopT 
ment ^ . 
Info- 
rma- 
tion 
Sel- 
ect- 
ion 


3.1.1 Act-«^ 
ion Types ■ 




19 




a. 2. 4 # Ru-^^ 
US Formed 


§29 


36 


28 


3.1.2 Des- 
ign Gonsid.. 


3J 


26 

e: 


21 


Seco-' 
ndary 
Per- 
f qr-^ 
naince 


1.3.1 # Re- 
lationships 


119 


161 


?5 


3.1 » 3 Res- 
earcl).' 'Info. 


25 


34 


24 


1.3.2 # Di- 
spositions 




111 


— 53F 

86 


3.1.4 Gen- 
eral Iiifo-. 


13 


19 


14 


1-5.3 # Ve- 
rbal Notat^ 




8 


11 


ALL 


\99 


.98 


72 


I'.K^ #• Gr- 
aphic Not. 




•4 


8 


^.1 

Solu- • 
tion 
Desc- 
ript- 
ion 
Sel- 
ect-^ 
ion *" 


4.1.1 Ele-' 
ment Glass 


• 18 


17 


6 


Comp- 
arat- 
ive 

Per- ^ - 
for-/ ; 
[Tiancfe ^ 


I.f .1 # Se- 
lect. Group 


66 


-65 


43 


4.1.. 2 Solu- 
tion Concpt 


' 32 


20 




1.4.2?^ Re- 
use Groups " 


45 


79 


52 


4.1. .3 Gra- 
phic El em. ' 


4 


. 6 


.5 


Ir.A-. J Aver.' 
Sel. Gp. Sz 


4 


4 


6 


4.1.4 Ele- 
ment. Specs 


6 


9 


2 


Aver. 
Reu . Gp . Sz 




. 3 


'3 


ALL . 

• * 


■59 


52 


49 

■CI 



-1 > 



•Underlines indiC^ife^'^gree of significance, 1 = .10, 
2= .D5, '3 = .jOj. ^ ' 

* indicates a/nigjier ^level of significance, on qne or 
more individual problems, than shqwn by underline. 



Table 5-2* continued 





MEASURE 


STRATE(^Y 
5 1.0.15 


Solu- 
tion • 
Ele- ^ 


5 1/1.1 'Mat--- 
e>ials 


23 


^3 


■ '8 


5^1/2 /Fur- 
niture V '! 


11 


14^ 


.8 


cnent 
Sel^ ' 
ect- f * 

italic 


5.1. 5. Equ- 
ipment . , 


16 


:i7 


8 


Anc- 
illary 


- ' ■* 

■ 14 




^0 




ALL ^ -^^ 


- 6,4 




34' 


ini- , 

tiori: 


lem^ °De scrip 


r ■ 


■v 




2.2.2"Prbc-. 

ess iBegcrip 


' 5 


0 


'4 


Info- • 

tion 
Re- * 
used ^ 


2. -2.. 3 Over- 
all Eeq. ' • . 




8 




2.,2.1-v Gei;A 
eral\Keq. 


0 


2 


, -3 






16 


10 


■'33- 


5.2 , 

Dgv- 

eloj:?- . 

^-'ent » 


3.2.1 Act-' 
io*i TjToe's ' 


16 


29 




3.2.2 D&G- 
ip;n Consid,. 


14 


24 
• \. 


27 


^Info- . 

I i . o 

tion 
Rg- 

uced 


3.2*3 4-<es- 
oarch Info. 


'0 


22 


✓1 


3 .2 ,^1- Gen- 
eral Info." 


O 


* 

11 


12 






.38 


90 




Solu- 
tion •■ , 
De sc- 


4.2.1 .Kle- 
nient Glass 


16 


.22 


4 


4". 2. 2 Solu- 
tion*"' Con6"ot 






* 

62 


ript-'^ 
ion 

I:e~ . 
used' 


4^2.3 Qra- 
■••nip jiileri. ' 


1 




2 

9 


ment b* pes ^ 


2 


• 6 


i 

1 






^— ^ — ^ 







.TYPE; ^ 


' /.MEASURE 


STRATEGY ' 
5 '10 15 


— ^ — 

5.2 

Solu- 
tion 
Ele- 
ment 
Re- 
used 


5.2.1 Mat- 
erials ■ 


6 


5 


.2 


" — S ="~ ■ ' 

5.2.2 Fur- ■ 

niture . 


••.9 


19 


^7' 


5.2.3'.Eq^3- 

ipment 


' 12 


.13 


..5 
— ^ 


5.2.4 Anc- • 

illary 


■ * 
11 


•8 


6 


ALL , . ■ -;• 


38 


48 


20 


6.1 

Solu- 
tion 
Scope - 


6.1..1 All 
Sol*. Items 


,124 


108 


83 


6.1.°2 # So- 
lution -Spec 


: 55 


36 


28 


6i1.3 # Dr- 
awing "bet' ■ 


'14 


16- 




5.2 

Soiu-,;. 
tion 
Para- 
naetr 
ers ' 

V 


6,;^;i # sq- 

are Peet 


. 11 


27 


13 


6;2.2 # Ro- 
om* Division 


4 


: 4 


* 


6^2.3 ¥ Fu- 
nction Area 




. -7- 


-9; 


6.2'.4 #• Le- 
vel Gliaii{^es 


"2 


,2 


•2 


7.1 1 

Solu-^ ■ 
tion . 
Eval- 
uat-.- 
ion 

V 


7.1.1 .Gom- 
prejiensive " 


' ;6 


■ 5 


5 


7.1,2 Eff- 
ective ■ 


6 


5 


5 


7.1.3' Eff- 
icient '. • 


V 


5 


.'5 


7.1.4 Aes"^ 
thetic ' . 


5 


5 


5 


7.1.5 Inn- 
ovat'ive 


6 


. 




7.1.6. Gomi- 
plete 




■5 


' - 4 


AlZi 


55 


32 


' 28 
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strategies acting over the three problems; the interactions 
between strategies and test problems; the individual test 
problems themselves; and .the variability of the subjects 
within strategy groups. It should be noted that since the 
test problems were presented to all subjects in the sAme or- 
der, the effects ascribed to problems may also represent 
sequence effects. As an additional test, the data for indi- 
vidual test problems was used to determine levels of signi- 
ficance for the effects of strategies on individual problems. 
However, it was assumed that these results would be lessv 
important due to the smaller amount of data involved. 



Figure 5-1 ^^atrix for Analysis of Variance 
(indicating data configuration foi; 27 data records) 



SUBJECTS 




SI 


S2. 


S3 






Xll 


XI? 


X13 




X21 


X22 


X23 




X31 


X32 


X33 






PI 


Xlll 


XI 12 


XI 13 


Sll 


X211 


X212 


X213 


S21 


X311 


X312 


X313 


S31 


PI 


P2 


XI 21 


X122 


XI 2 3 


SI 2 


X221- 


X222 


X223 


S2 2 


X321 


X322 


X323 


S3 2 


P2 


P3 


X131 


Xl^2 


XI 33 


S13 


X2'3l 


.X232 


X233 


S23 


X331 


X332 


X333 


S33 


P3 




Xll 


Xl,2 


X13 


SI 


X21 


X2 2 


X23 


S2 


X31 


X32 


X33 


S3 


X 



MODEL: Xijk = X + Si + '(SP)ij -f Pj + Uk(i) > (UP)kj(i) 



Xijk = individual score on a given measuire 
X = overall average ^ '/ 

Si f test strategy (SI = 10-step; S2 = 15-step; S3 = 5~^tep) 
Pj = test problem. (PI = Office; P2 ^ Snack Bar; P3 - Conf. RiTi-) 
(SP)ij = strategy-problem interaction / 
yk(i) = subjects within strategies / ^ 

(UP)kj (i) =\Error ' / , 

Xik = individual subject losing a given! strategy 
Xik = average score for individual sub/ject over all problems 
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Sij,= average scojjeOfor each strategy over all problems 
S^ij =. average scpre for each strategy on a single problem 
Pj = average score for each problem over all strategies 

C. Significance of Results ^ 

The outcome of the analysis of variance is suxamarizecl 
in Table 5-3^ and the respective levels of significance are. 
indicated on T^ble 5-2 and on the individual results tables ^ 
in Appendix 1-2. The most important observation, is that 
even including significance, at the .10' level, the overall 
strategy effect. was demonstrated on only 8 out of 71 measures 



Table 5-3 Distribution of Significance 

(figures indicate number of measures which display a giyen 
level of significant difference for each variable) a;: 



SIGNIFICANCE 


• 


EXPERIMENTAL VARIABLES 




' 


Strategy 
Overall 


Strategy 
/Problem 


Strategy 
^Problem 


Ml 

Strategy 


Problem 


Subjects 
il/ln St. 


.61 


2 


2 . 


8 [ 


12 


13 


30 


.05^ 


4 


6 


10 


20 


7 


16 


.10 


2 


. '8 


• 6 

a 


16 


12" 


" 4 


EST 


(12) 


(0) . 

A 


.(0) • 


(12), • 


. (0) 


(0) 


ALL 


20 


16 


24 


41* / 


32 


^50 



NOTE: "EST"' represents measures which displayed consistent 

strategy overall differences based on inspection of the 
data, bat below statistical significance 

^indicates total number of measures which displayed strategy 
differences, adjusted for ov^erlapping variables 



This result is statistically within'the area of a random dis- 
tribution of effects. Additional strategy effects were 
shown for indi. dual problems, tjut again the number is small 
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15 

as compared with the total measures . The effects of strategy 
interaction with problems were somewhat more numerous, sug- 
,ge.sting the possibilit^of some difference between strategy 
groups. However, these interaction effects are difficult to 
define and still affect only a minority of the measures. 
This overall low level Of significance is modified slightly 
by a grouping of the different types of strategy significance 
with measures where some effects were estimated but not 
statistically significant. This grouping includes over half 
the measures but at best can only serve as an indicator ^ 
possible significance confounded by other .experimental 
variables. In particular , ^ the effects of individual problems 
and variability of subjects within the strategy groups appear 
to have had as much if not greater influence. 

The lack of significant strategy effects is further 
emphasized by referring to Table 5-2 and noting the lack of 
significant differences for most 'of the overall performance . 
measures in category #1, and the absence of significance for 
solution measures in categories #6 .and #7. Even in measure 
1.1.3 # Frames Required (by definition, twice the number of * 
minutes required for problem-solving) , the apparent signifi- 
cant difference in time was based on a highly abbreviated third 
"test session by the 10-step strategy group, attributed to sub- 
. ject misunderstanding of the experimental task. The strate- 
'gies therefore did not appear to demonstrate effects on the 
measures of overall information use, amount of time or effort 
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n 



required for solution, nor were the solutions produced by 
the different strategies significantly different. The ini- 
tial, basic conclusion is that the experiment was not suc- 
cessful in demonstrating comparative differences between 
strategies • 

EXAMINATION OF MEASURES 

Each of the measures was examined^^by inspection, aftfer 

the initial analysis of variance, to determine if any sug- 

' * v» ' 

gestive differences were present that might contribute- to 

' . • . \.. , 

a better understanding o:^ the results. The examination con- 
centrated on measures where some level of significahce Was 
present, but also included measures where consistent dif- 
ferences could be observed by ignoring the scores of at most 

one subject in each strategy group. This latter approach 

i 

represented even lower levels of signif iqance, but did make 
some allowance for' the high variability in subjects. In '^con- 
ducting the examination, the .emphasis was on interpreting 
apparent differenced in terms of the meaning of the measures 
and their interaction with the experimen^tal variables. This 
was largely based on the experimenter's experience in develop- 
ing the experimental materials an.d a general background in 
environmental design. ' The purpose of the examination was not 
to fgrce conclusions in the absence of meaningful signif icanc.e 
but rather was included to demonstrate some of the capabili- » 
ties of the experimental me tho'd 'and raise some questions for - 
further experimentation, ^ . * " ^ * • 



A, Problem Results . ■ 

The effects of the test problems on different measures v 
were important because of the attempt to control the selec- 
tion of problems in order to minimize problem effects, and 
because problem effects operated across strategy groups 

providing a basis for comparing specific strategy differ- 

' ' ' ■ . - 

ences. The differences in average problem scores are shown 
in the individual measure tablesi in Appendix 1-2, ^but the 
summary of types of effects for different numbers of measures . 
is provided in Table 5-4. This table indicates problem^ ef- . 
f ects . in various different configurations. The most preva- 
lent configuration was a descrease in the scores between 
• prpblems^. This decreasing effect does not' appear/to be due. 
to the problems themselves, since there is no ^olfvious tie- 
in with problem characteristics , but rather appears to indi- 
cate a sequence effect* over the three test sessions. The 
assumption is that the subjects became more selective in their 
utilization of "information arid related activities due to an 
increased f ami liarity:.' with the experimental task\ However, 
this<?may also represent a lower level of motivation with^ 
repetitions of a similar task. Specific measures demon-^ 
strating this effect and possible interpretations -are given 
below: • ' - ' ^ 

1,1,2 # Cards Gener a ted - shows a steady drop between each. of 
the three test sessions .(p=.01), perhaps due to greater ef- 
ficiency in information handling. ^ . • . ' 



Table 5-4 Distribution of Interaction Effects 



(figures indicate number of measures on which a given variable 
displayed an effect with a given configuration) * 



CONFIGURATION 


. VARIABLES' 


Type 


Session 


Problem 


Strategy Perform. 


Strategy Solution 










10 


15 


5 


10 


15 


i-iii„ 


All 

oueaay ^ 


■ 7 


6* 


0 


1* 


0 


0 


. 3* 


II 

Down 


4* 


2 " 


3 


13* 


0 


, 0' 


0 


II 

Up ■ . 


8* 


■ ;3 


5 


5 


0 


0 


0 


ALL 


19* 


11*; 


8 ■ 


19* 


■0 

— »- 


0 

< 


'3* 


— 

DECREAS- 
ING 


I-II 
*ijown 


■ 16* 


■ 3* 


10* 


0 ■ 


4* 


0 . - 


0 - 


II-III 

Down ^ 


8* 


2* 


' 6 


2* 


0 


0 


0 


i-ii-^ii 

Down 


9* 


. 1^ 

• 


8* 


0 


• '1* 


0 


d 


ALL' 


■33*. 


6* 


24* 


2* 


6* 


0 


0^ ■ 


'r 

■ INCREAS- 
ING 


I-II 

up' 


■2* 


0 




- 1*' 


0 


3* ■ 


0 


II~III 

up 


0 


' - 0 


0 


0 


0 


0 


2* 


I-II-III 
up 


1* 


0 


0 


■0 


0 


2* 


0 


ALL 


3* 


0 

> 


0, ■ 


1* 


0 


5*, 


2* 



NOTE: Interaction effects occur .When a" gisoVen variable influ- 
eni:es th,e data in definite direction over the three 
test sessions (I, II, III); for strategies the inter- 
action represents a direction taken. by one strategy, 
while, the other two strategies move in a diffe'rent V 
direction . , 

* indicates that a majority of the measure displayed a signi- 
ficant ii:iteraction effect at least p=,10 

For Strategy Solution, a decreasing direction indicates that 
the solutions a're improving * * 
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1,1,3" # Frames Required - indicates a major drop between the 

secbnd and third session3 (p=.10), suggesting that although 

the level of information processing decreased from session i 

o * ■ . 

to II, the discrimination required additional time. 

The individual measure groups reinforce the decreasing pat- 



tern with 3.1>5 All Development Information Selected arid 4 > 1 . 5 
All Solution Description ^Information Selected showing a steady 
decline over three problems (both, p==,10). In addition -the ' 

drop in Frames is para lied by 3.2.5 All Development Information 

'..1 . 

Reused ( p= . i 5 ) . 

■ * . 

. There was also a noticeably different response' given on 
problem II, apparently due to sub ject'. interpretation of the 
Snack Bar as a larger scale problejfi than the other two. The 
effect i^s» first noticed on 2.2.3 pv^'rall Requirement -Informa- 
tion Reused , which increased in fre^quency in comparison to 
problems I and II, although not significantly, suggesting' 
that more effort was required to define the more complex 
problem. This effect is reiriforced by. a similar pattern fo3 
4.1.2 Solution Concept Information Selected , apparently indi- 
eating a corresponding increase in solution complexity • The 
assumption* is also supported by a decrease in 5.1.2 Furniture 
Elements Selected and 5 . Iw 3 Equipment Elements Selected (p=..bl, 
.10)/ wh^ch might.be expected where these elements were less 
important" at the la^'rger, scale . The scale assumption is 
literally confirmed by the relatively hiaher levels shown in . 
^ 6..1.3 # Drawing Details^ j6. 2. 1 # Square F^t ,. 6.2.2 I' Room 



Divisions , 6.2.3 # Functional Areas (p=,05, •10, .10, .05)|. 
B. Pe rformance Results , ■ 

— " — . ■ *' 

The effects of strategies bn problem- solving performa; 

while lacking strong significance, did display a number oi 

diffiBrences which could be related to the characteristics of 

the specif ic strategies , The effects- for strategies lover 

M ■ i * 

all problems are indicated in the aiverage scores shown on 
Table .5-2: Strategy-problem interactions are based on N|:he 
strategy averages for each problem 'shown in Appendix 1-2, 
and key relationships are represented in Figure 5-2 and in 
Appendix 1-3. The range of strategy-problem interaction con- 
figurations is ^shown on Table 5-4. Important measures re- 
lated to pverall performance, problem-solving information, 
and solution element information are described below with 
possible interpretations: 

1, Overall. Performance Measures (Category #1) 

These indicate a number of .strategy differences affecting 
overall performance, demonstrating different basic strategy. 
Qipproaches. . * 

^ i.1.2 # Cards Generated - The 10-step strateigy tended to 
generate more data cards than the other strategies on all 
three p±obreins, although ^^th^ effect wa*^ only i^ignif icant on 
probleii\ .r (p=.10). This represents a high level of activity 
on all measures. Similar effectg appeared on 1.3.2 # Disposi- 
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tions , (p=^.05)/ reflecting a large amount of activity with 
individual item cards; and 1.2.3 # Groups -Formed (I, p=.10), 
indicating a large selection and reuse of items (see 1.4.3, 
l.,4.4) . The impression is that the lO-step strategy tends to 
encourage greater concern .with detail in handling information 

The effect of the 10-step strategy is also apparent in 
the problem interaction on this measure, with the 10-step ' 
showing- a dramatic decrease in the number of cards used/.- as 
much as 300 cards between each session, while th^ other stra- 
tegies remained steady (p= . 05) . Thrs pattern is repeated on 
# Dispositions , #. Groups Formed , and 1.2.4 # Runs Formed 
(p=.-05, .10, .10) . The pattern is particularly evident for 
3.1.5 All Development Information Selected (p^-.Ol). The 
suggestion is that some i^nitial confusion resulted in very 
high activity on the first problem, with a second re-evalu- . 

ation taken after the second problem. 

' .- 
The 15-step strategy demonstrates an- alternative inter- 
action effect on this measijre, with a slight but noticeable 
drop in activity between problems I and II and then a rise 
back up again on problem III.^ This pattern generally occurs 
on various measures in parallel with the 10-step steady de- 
crease, particularly on Dispositions , Groups , and Runs . 
Examples of the pattern on specific measures include: 3.1.2 . 
Design Consideration, Information Selectic/ ^ p= . 0 1 ) ; 3.2 .4 
General Information Reused ; and 3.2.5 All Development Infor- 
mation Reused (p-.0 5) In comparison with the "Problem 
Results" , these suggest that the low number of. data cards for 
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the 15~step on problem II may be due a sensitivity to 
scale with a simpler development usee J If or the larger scale 
problem. " ;^ I 

1.1,1 # Information Items Selected y- The drop in overall 

activity for the.lO-step strategy is Repeated on this measure 

between problems I and II, with the other strategies steady 

'(p=.01) . This reinforces the assumption that the 10-step . ^ 

group made an essentially' random selection of information on 

the first problem^ which was . corrected on p.rob'lem 11. A 

similar pattern occurrs on 1.4.1 # Selection Groups Formed 

(p=-05),. and the area of impact is indicated by the^^on- 

significant but:> distinct repetition of the pattern on 2.1.5 

? ■ ■ ' • . ■• ■ ' f " — 

^ - All Definition Information Selected ] and 3.1.5 .All Development - 

Information Selected. ' I • 



\ 1,1.3 # Frames Generated - The 5-step strategy required noti- 
ceably more time to complete the design task/ particularly 
IS compared with the 10-ste^ stra^pgy (p= . 01) . f In comparison 



at 

to 1.1. 2/. this suggests th4|||fthe*5|- step strategy^^w^ less 
efficient, requiring more time to jaccomplish less information 
processing. 

The interaction effect on this measurfe shows that the 
15-step strategy remained relatively steady over all three 
. problems, while the 5 and lO-steti> strategies showed a major 
^decrease in frames between II and III (p=.01). Since the 
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frames for the 15--step were initially low, it is possible 

that the other strategies were developing more efficient 

procedures over the three sessions. 

This assumption is reinforced in part by 1.3.4 # Graphic 

■a 

Notations / where the 15-step was noticeably high on problem I 
(p=,05), despite experimenter attempts to limit graphics in 
favor of a " coordinate" system. The suggestion is that the 
15"Step group completed highly developed solutions on I,.whicj;i 
required grapl:iic expression. ' In addition, 1,3.3 # Verbal 
Notations showed a relatively high level for the 15-step on 
problem III (p=.10), raising the possibility that the 5 and 

10-step groups oversimplified their procedures by the third 

I • ■■ ) ■ •■ ' . 

problem, while the 15-step was std^l^ operating at high levels 

of complexity. ' ' • " 

2. Problem-Solving Information (Categories # 2-4) 

These measures indicate different 'responses to informa- 
tion types or categories, including both selection and reuse . 
of . information in each session^ Specific information differ- 
ences suggest clues to ba^ic strategy philosophy and approach. 

2>1>1 Probxem Descriptor Information Selected - The lO'-step 
strategy was low overall on this measure, particularly for 
problem III (p=,10) . This type of information w/as used to 
define the problem in general terms before beginning the 
detailed definition, and the low utilization suggests ' a less 
structured approach to problem definition. • The effect is 
reinforced b^ 2.1.3 Overall Requirement Information'. Selected , 

,•, ■ , ■ ^ ■ . ■ . 1- ■. ■ ■ 

. ' ; ■ .■ , - ■ :94 , . .. > , . 



which was very little used by the" 10-step group on problem 
I'll (p=.10), apparently reflecting an oversimplified deifini- 
tion on this problem. In contrast, the 15-^step group was " 
noticeably high pverall on 2.2.1 Problem Descriptor ginforma- - 
tion Reuse , particylarly aS'compared with- the nimibef of these . 
items selected, suggesting a greater concern with formulating 



the initial definition, 



2.1. '2 Process Descriptor Information Selected - The ' 5-step 

strategy was very high on this measure, with the .other stra- 

. ■ . ... .. .. ^ 

tegies showing negligible scores (p=.05). This information . 

type was unique,' listing alternative design "techniques" o^p. 
"heuristics" rather than problem information, and the assump- 
tion i's that the ►S-step group was uncomfortable with* their 
open-ended strategy and used "this infor:mation to further. 

' . . . , ^ . . ' y/ - • ^ 

clarify . their design aqtivi ties . ' ' ^ ^ • :^ 

A parallel effect* showed- the 5~step- group with a signi- 

ficaritly high score on 2.1.5 All Definition JnlEormatioh Se- 

lected (p=.01), suggesting that in ge'neral the strategy used 

a. more diffuse search process in attempting to defirie the prob 

lem. • This is* supported by the\^igh* leve*! shown on 2.1.4 

General .Requirement Information Selected (Illy p=.05),o an 

information type which was considered to be marginally rele- 

y ... ^ • . ' . . 

vant, and was included in the information bank primarily for - 

■ " . . — " ' « ' ■ 

the gake o.f completeness-. • ' . - 

■ ■ . ■• . • .\ 




3,1,1, Action Type Information Selected '7 Both 5 and 10-step 

strategies wer.e high overall on this measure^ with the 5- 

step particularly high (p=. 05). This effect is impJortant 

since this information type simply lists a? terna'tive "activites 

_ which might take place in" the. problem environment, ami con-- ^ 

centra'tion on an abtivity~by-activity .analysis • seems to.be 

< * • , <^ 

^ an; inefficient'/ niechanical approach in a limi ted pifob'lem-solv- 
ing session, ""it should"*be . noted that the .lO-step groug)* tended 
; * > to ;be Wiry high . on Reuse (3^2.1) of this information, Indi- 
t^Vr'-ci^tin^ that the items received more than ■ casual' a.ttention. 4 

If iX^^^JSra^m^ effect -is -evident on -4. 1.1 Element Class Informa-;, 

^^r':r\^H\^.- ^ ^ r ^ ^ — , 

;€ife)[^ii-^^i^-i^'i4ci^ed , with both' s-tr-ategies relatd^^^ely higH on this 
•^listing' o'jf alternative general types . of, solHition elemeh^ ^^ 
uggesti^ng a 'more 'dogged, ■Empirical approach: 

4.1,2 Solut ion Concept Ihform'atioh Selected - -In contras-t with 
■ — ^» — , , — V , 

• * ■ 1 ■ • . , / ^ 

the above, the 10-step -stratei'gy was low overall) on .this mea'- - 

sure (p.= . lt).^) , which includes- ''GonceptUal statements describing 

variety- of sofution approaches and diVen^ions . This* "irif orma- 

-tioh. allowed suJpjects to sketch out solutions in general terms^ 

—before .b^eginning detailed development , and the low .utilization 

confirms the more pedestrian appipoach . . Although the. 5-step 

<s o . ' ■ • :^ . . / ■ ^ ^ - y - 

strategy was high on thi,s'*'measure , 4, 2. '2 Solution .Concept 
— Itrft^rma-feLgn Re us e d , shows a much higher 'level of application 

• ^ ^ - / ; 

' ■ • " * ♦* , * * * * 

for th^ 15-step^^rqup. In particular, a^ s'igijif icantly high 
score for Reuse on probletm I ,(p-.10) suggested that.^the 15- 

••■•>■./ i .■ ' 

* ■ . H . i ■ ^ * 



stQp subjects w^re able to comprehend and utilize •thi'S"* infor- 
• . 1 ^ - . . . , ^ . ' V 

mation muclx earlier than the othe^r strate'gies. . . ' ' 

3>1>5 All Development Information Selected - The 15-step • 
•group was generally low overall on t^is measure, although 
high subject variability preA^ented significance. Thfe ^ssump- 

• " J 

tioi\ IS that the *15-;^ep approach was' more Economical and 

* • ■ 

selective of irjformation used in. development, with a high ^ \ 
depe'ndei^ce on overall conceptual i-nformatiqn. In comparison, 
the 10-s tep stratisgy was high on this measure, and particu- 
Tarly high on 3>2.5 All Development > Iriformat.ion Reused ^ 
^4p~ -05) , ^ and 3>2>4 General Information Reused (II, p= . lO ) . 

The impress'ion is that the lOrstep" spent qonsi'derable amoUn^ 

■ f- . ■ . 

of effort on "busy work", since the "research" information 
was only marginally useable given"" the time and scale limita- 



tions ; and the "general" info.rm"ation Jjfas largely concerned ^ 

with, previous^ traditional solutions. 'It should be -noted 
N ' * . ' ' • 

that ^the 10-step group did not make correspondingly high use 

■ ■* 

of Design Consideration , Information 3 .1.2, 3.2.2), considered 
by the experimenter to be the most relevant design development 
informati^on. ' . • 

3. Solution Elements (Category #5) ^ * * ' 

L ' ■ " 

Although the selecrian and reuse of solution element 
information did not show many significant strategy effects, ' 
there were some indications of differences in this area. 
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5.x. 2 Furniture -^Elements- Sie'lected - The IS-^tep strategy 
displayed the sensitivity to. problem effect 'mentioned earlier 

/ ^ ' . ' ' . * ■■■ 

with a noticeably low score on this measure for problem II, 
and an increase on III, while the other strategies remained . 
the same ""be twee n problems] II and III. (p='.05), presumeably. 
de-emph^siziing furniture at the larger scale, Thi§. <assump-. 
tion is supported by 5 2' Furniture Elements Reused , on'which 
the IS^step had ^an^ Average score of "0"'^on II. 

' ^ i / . ^ ^ . 

An vinverse effect for this strategy- was noted ^on ,3 > 1 > 3 
* *■ . • , • 

. Research I'nf ormottion Selected , 'where" the IS-step was notice- 

ably higl^er on 11 , presumeably indicating layout and perceptual 

• , * ' - 

information •applied -to the problem. Another indication of ^ 

i ^ ' . ■ ■ ■■. ■ . 

.sensitivity diffe're^ces was observed in the interactions on 

4 . 2 ..2 -Solution Concept Information R,eused , where the 15-step 

used fewe|: conceptci on II, while both the 5 and 10-step' 

groups increased their useage on II. Appatently/ thei other 

« * » 

•atrategies responded' to the change in scale by making greater 
^use of more general concepts ^ while the 15-step simplified its 
•approach (Note: 'the 5 and 10-step strategies showed only 
minoir "serisitivity'V responses , ^ee Table 5-3) . ^ ; 'v 

5'. 1.4 Ancillary' Elements Selected - Th^e 5-step strategy \ 
displayed significant intferaction effects on this meaisure^ 
^ increasing i^ use between all three probleit^s ^ while the ' 
'mother strategies were steady or decreasing (p=.05). This is 
made particularly evident" by significantly higher levels for 



the 5*-step oii problem'lll for both Selection and ReusQ 

(5.2.4) (p=.05/ .10).\ The information .type represents a eon- 

cern with spat^-al enclosure and -mechanical systems , less 

central to solution development^ and the assumption is that' 

'1!l|||^5-step group ignorjed or iT)inimi^e(i these aspects due to 

uncertainti'es about solution formulation on the earlier prob'-^^ 

lems, and later Increased in"' oonf idence and was able to o 

explore the solution in greater depth • * 

' ' ^ • \>. . . ^ . 

C.' Solution Results 

The .two -attempts to c9mpare solations , by mfeapures of 

different* chai^acjggiiristicsr and by* expert evaluation, had 

been intended to demonstrate differential strategy^' effects . 

-However, the results "indicated consideral^le^ uniformity Ijetween 

n' t" » 

strategy groups, particularly- in comparison with the^ variations 

' in. problem- i^l'^ving performance. .The examination in this area, 

therefore, .i5. based! on the pcxssibleMmplicatidns of relatively' 

j^slight differences between scores. - 

1. Solution Characteristics j[C^tegory #6)' " ' , ' 

These measures were developed- in an attempt to identify. 
"* /■ 
objective differences- in; the sol;ation approaches takeA by 

"* * ' ^ * ' 

sxibjects in different strategy groups.. Due to the general 

uniformity in solution drawings and- specif icati^ns,. there 

'were very few meaningful' differences in this atea: A minor ^ 

interaction did occur on 6.1.3 # Drawing Details where . the 

5~step strategy remained steady while this others strategies 



increased in detail on II , . presumeably in response to scale. 
Another measure, 6.2.2 # Room Divas ions showed^Jtlie 15-step . 
.stfateg|^ ^^-^^i^ problem III (p=%05) suggesting the* main- , 
tenance o£ a high level of complexity on the final problem. 
2. Solution, Evaluation (Category #7) 

V* . The -solution evaluation measures,: .gen<grated by, a pro-- 
fessional jury of evalua^tiorsv were intended to provide a 
comprehensive comparison between the solutions produced by 
subjects in different strategy groups^ However, the re- 

surts of the evaluation weVe~ inconclusive, the ^valuators 

I f '* ■ ' ■ ■ ■ 

themselves^ were highly variable and the re'Bults of their . 

evaluation ^as shown in Table 5-2*and Appendix 1-2 averaged 

out to very similar scores^ well below significance. Despite 

the^ questionable significance, observation of the data does 

suggest that the 15-step, group may havco produced slightly 

better solutions, both in terms of all .criteria: 7>2.1 All 

Criteria Ranking and Rating Combined ; . 7.2.2 All Ranking jcri - 

teria; and 7.3.3 All Rating Criteria ; 'and specific measiires: 

* ■ ■ ^ : . • • i ■ • .■ .-. ■ ^ ■ ■ / 

7.1.2 Effective , 7 . 1 » 5 .Innovative / ' and i7.1«6 Complete . Thus 
there is some suggestion that the 15-step strategy is more . 
useful; providing effective, workable solutions, although 
not iiecessarily ^superior in comprehensliveness'* or efficiency ; 

maintaining an innovative solution appifoach (as stressed in 

^ '/ ■ : • * ; ■ . ^ ' ■ " \ * 

t^le' problem statement); an9 maintaining a complete, thorx)ugh 
/I ■ " ^ ^ 

'/ ' ■ ■* i ' . ■ ^ ' ' ^ • 

solution spe*crf ication.. " • 
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■ ^ t ■ • ' 

^ Interacti,ons on the evaluation i were significant (Figure 

«* 

5-2) r with the S-st^p strategy showing a definite improve- 
• ment Over the three problems (fpr all criteria) , while the 10- 
step showed worse scores dfter the first problem, and the 15- 
step was steady to slightly worseon the last problem (p=.01). 
-The assumption is that. the 5-step demonstrated a learning 
: effects while the' 10-step produced acceptable results only 
with the greatest level of activity. ThelS-step displayed 
rela^^ve consistency after .starting rather successfully, and 
the drop in quality on the final problem ma^ have been due ' 
to the evaluators misinterpreting the high level of solution 
complexity which was • continued by this strategy (in accordance 
with experimenter' s definition ^of the problem), whil^e the 
other strategies simplified their fi^al solutions . 

\^i D, Subject Results ' : 

. * - The variability, of subjects witfiin strategy groups 

' apparently was a ma'jor factor in lowering ^le ieyel of signi- 

ficance for the strategy effects. iThe extent of the - vaTiabil-■ 
v , f • . . , ■ ■ ^ • ^ ^ ' ■ 

r /ity is shown 'in Table 5-5, which indicates that all subjects 

■\ . ' ■ . ^- . ■ . 'V' : • 

' \ varied uniquely from the other members in their groups on 

at^ least so|le^ measures, and that the directi9n of the dif:^er- 

ence in 'frequency scores- and evaluation scores was sel<iom con- 

sistent. This suggests a'high level of essentially random, 

^ *personalized^^in^^erpretations of the problem and strategy 

< within individual sub jects /"^Howeve^ an examination of , some 



101 



I . 



92 



specific subject' behaviors may help in .understanding some of 
the .Strategy: group scores: ' j ' 

^ A particular subjeqt effect was apparent in iihe evalu- 
ation of the solutions. For all crit^rl^a^ subjects within 
strategies were significantly diffeifent (p=.piKr"in p^rticu- 
lar, two of Hhe subjects using the 15- step strategy were con- 
sidered to be "very successful . (compared with all other sub-^ 
jects , With only one subject, in t-he 5-stepi g[roup as success- 
ful), vjhile the third subject' scored vety poorly also in 
comparison with all other subjects), 'it is possible that ^ 
the two subjects actually represented the effectiveness of 
the strategy, while the third subject ' twith an unusual 'ap- 



proach, described below) was misinterpreted by the e^aluators. 
Table 5-5 Distribution of Subject. Effects 



STRATEGY PERFORMANCE ~^ 


Difference 


^ 5-Step 


. 10-Ste] 




IS-Ste 






1 


2 


3 


1 


2 


3 


\1 


2 


3 


HIGH ' 
LOW •» - . 


3* 
3* 


3* 
9* 


19* 
. 3* 


2* 
2* 


5 _ 
4* 


21* 
6 


4* 

1* 


' 6* 


12* 

J 

15*. 



STRATEGY SOLUTION " 


Difference 


5-Step . ' . 10-Ste] 




15-Ste 


p 


1 


1 


2 ■ 


'3 


1 ' 


2 


3- 


- 1 


2 


3 - ■ 


HIGH ^ 
, LOW 


I ]_ * 
0 


1*. 

.0 


1 

0 

1* 


1* 

. o' 


"0 
0 


0 
0 


0 

1 5* 


• f 

0 

3* 


.5* 
0 



NOTE: *^ indicates that the majority of the measures^isplayed 
. .-a significant subject Effect at least p = .10 

For iStrategy Solution* a iow^ sgor-e indicates a "better" 
solution , ° . ' I ' , ■ 
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^Tt^e third subject in thj^e^lS-step group also displayed 
unique and significant differences in selection and reuse of. 
two major types of inf orn)a.tion, scoring very high on prol^leTri- 
relate^a information (generally p= . 05) , wh^-le- having^ low or 
negligible , Scores (averaging 4 items to the group average o>f 
52) ^fof solution element inforpation (p=.bl). While this may 
be- explained in part by an o\/er-concern with problem develop- 
ment:; the accompanying solution graphics suggest a very* com- 
prehensive solution rfencJered in* genexal term's. * *' ^ 

^ AdcJitional subject effects indicate- general strategy 

group trends, consistent in themselves, carried' to e'xtremes 

. -■ ^ . ^ ■ ■ ^ - . 

by. individual sub^jects.. Thus, one sub ject- in the 10-step 

. / ^ ^ . 

group showed an extremely 'high reu^e level particularly for 

various types of deve<Lopment information (usually p=.01); 

^ while one s-ubject in the 5-step group displayed consistently 

high levels of selection fdr all information types (usually 

p=.01)'. These confirm map or strategy effects and suggest 

ways ,in which indivi-dijals might have disallowed significant 

effects . ' , > ^' ' 

^ - . ■ • ■ • 

E. Debrietfing Results , * ■ , . • 

The. (Jebrief ing session, after the running of the experi- 
ment, offered the only introspietctive response from the siib- 
jects and was directed to r^esponses to ^ihe 'experimental for-, 
mat rather than the effect of strat^agies. In general, the 
responses shown in Appendix E-2, suggested that thei^^experi- 

' , . - los ■ 
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mental taskWas accdpted for , the mosl^' part ' by the subjects. 

The basic approach, using» the information bank and test^ap-. 

parcit,us, seemed^ to be workable 'with minor revi&ions. Although 

. there were some reservations about the artificiality of the 

experimental situation, there. seemed, to be overall agreement,' 
• . ' '. \ V V ■ 

•s^ or at least lack 6f strong disagreement, that the experiment w^s 

able to accurately reflect design problem-solVing and utili5:al- 

* tion of strategies. • ■ - , 

^ CONCLUSIONS * ' ^' . . . ' 

' .* 
\ Tihe conclusions drawn from this experiment can only be 

general and suggestive. The high variability in the summary 

da,ta dpes not permit conclusive judgements aibout strategy dif- 

* ferences .v^The^larger ^masg 'of data on the details .of problem- ^. 
, solving ha^' not yet beem processed, and while this may add a^ 

^ J be^tter understanding of design activity, it is also likely to ' 
be highly variable. D^ippite the high experimental and subject 

> variability;/ tentative directions are discernible,, incjicatilig - 

'•^ ■ , - * - . ■ ' 

im^nications .for further research. (-An asterisk * after a word 

indicates that the statement did have statistical significance • 

I'east at the ,5%' level . ) • ' ' • * ' ' , 

^ " A. Suggestive Strategy-Differences ".r, . ^ ^ 

" . ' ' ^ • / " 

» ^ •Given that a statistically significant difference was not 

obtained, among the strategies, some actual differences observed 

iii the examination of measures can' be expressed as sugge^tiv^ 

* ■ ^ . • - r *' ; - 

; conclusions. They are specifically .compared to the .experi- 
■, i * ■ ' ' . ^ 

/ • * ■ ■ . ♦ ■ 

mental hypotheses listed on pages 38-3^. 
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1. General Hypothesis ' ' " 

The actual number of strategy Sifferenbes and- the' consis- * 
• . ■ , . , • ' f • ' ■ . 

tency of the 'differences In line with hypotheses about strategy 

• , , ) ^ ■ -I . ' , . . 

effe'cts sugg'est that strategy significance is masked by subject' 
variability. -^.Strategy ^differences are apparent throughout the 
average strategy scores (Table .5-2), and pronounced differences 
were evident in s trategy-probliem interactions (Table 5-4) . Although 
not "statistically significant, the differenefes are too numerous^ 
to ignore entirely. ' \ 

2. 5-.Step Strategy . ^ ' , 

■ ■ 4 a I ' 

An app^arent low leve^ of ef f ectivenessi^ for this ' strategy 

was demonstrated in "part by the -additional time required for 
' * * * * * . ' 

arriving at a solution.,, and the apparently random selection 

. / • ' ■ ■ • , . : < . ^- ■ ■ ■ . , ' 

'and app'^lication of information. -Overall, the su^qcts did ye- 
quire more time for solution,*, particularly as |Compared with the 
10-step strategy group (Hyp. 1-1).' The subjects did demonstrate 
an apparent .uncertainty ^Ln beginning problem-solving vy_th a <> 
relatively high selection* of definition information (Hyp. 1-2) • 
Although it was not anticipated, the \,extensive use df' process 

descriptors* to supplement . the strategy', steps , .cleatly disti.ng- 

. '•. '■* 

uished this strategy group. The quani:ity of inf9rmatipn selected 

■ ' . ; * . " ^ * 

by this group was not noticeably • higher than the lO-stef). strategy , 
although the "average number of selections were somewhat greater 
(Hyp. 1-3) . ' By contrast, this 3trategy did not -ishow 'much change 
in effici^ency. (decrease in selections) over time, "particularly 
as compared with the 10~step (Hyp^. •1-4). The solutions produced 
by this strategy o^d improve over time*, particularly between 
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prot)lems I, and II (Hyp. 1-5) , However,, the solution quality 
appeared competitive or superior to other strategies after the 
first problem. This s?trategy* (^id appear to opera'te better than 
had been anticipated, in many ways similar to the 10-^step, but 

some difficulty .was shown in handling in'formation and in the 

■ ^ I ■ 

*» • • 

development of solutions. ^ ' 

- ' ' . / " ■ 

3. lO-St-ep Strategy . ' ^ 

This strategy portrayed an apparent uneven performance 
with an overconceg"?! about details. This is suggested by the 
overall high leve?ls/bf acti)Vity. Little difficulty was noted 
in the initial phas'e of problem-solving with a relatively con- ' 
servative selection and ceuse of definition information (Hyp. 
2-1). In genera'l, this group displayed a considerable amount^ 
of activity with a high number of tiata cards generajted and a 

(5 ' . * 

high reuse of information, particularly in more peripheral type 
categories (Hyp. 2-2) . ; Subj^ts did show a low use o*f conceptual 
information, particularly as compared with overall ^^ligh levels 
;^^or otAer categ.ories (Hyp. 2-3).^ Improvement i^ efficiency 
(de'crease in activity) was clearly demonstrated* although, at much 
higher levels* th&n' anticipated, su-g^esting a ^superficial treat- i 
irtent in latet problems (Hyp. 2-4).. The data generally indicates 
low ](evels -of solution quality for this strategy, and inspection 
'Of solutions suggests their more conservative aspect (Hyp. 2^5). 
However., the| decrease in quality between .problems I and II was 
not anticipated, and suggests an additional fa^ctor of disillur , " 
sionment or overconf iden,ce-. This strategy then, .representing 
pi highly typical approach, 'is apparently ".work^le , but it 

^ ' ' lOG ■ 



produces ■ only a moderate level of 



» - 
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per'f ormanpe . * 

■ •. ' . . ' ' ^ . ^ \ • \^ : 

4, 15-Step Strategy * • / j 

This hypotheti<pally °bes t strategy was sli^Jitly^ but conV , < , 

• » *. 

sis tently' more effj-cient and effective in comparison witH^Vt;^e 
Other strategies,. These subjects appeared to establish ^ system-^, * 
atic definitions as indicated by a substantial selection of * \ ' 
problem'* des'criptors and a generally high level of definition 

activity (Hyp, 3-1). The subjects ciid indicate lower levelS'^of 

< • • . ■ ' . » 

. information selection in most categories, particularly those-. 

, less' -central to problem development ^Hyp• 3-2).. In contrast/' 

' ' - \ ■ ' ^ • . - ..'-'^ ' 

•the subjects did show high levels of selection, and u^e of conceptual 
information (Hy.p. 3-3). The subjects did remain '^relatively \ ^ 
steady on most measures, with allowance for the reisponse to 
. • scale differences on problem II (Hyp. 3-4). The' solution ratings 
• . w.ere slightly but consistently bett:er, particularly on .specific 
criteria such )as compre]:>ensi:^eness and innovatiyeness , which 
' had been stressed in ?he problem s tatfementi*. .. These results 

suggest thatthis is a definitely workable strategy with potential 
advantages that could be exploreql in. further study. 
• B. Research Approach i ^ , ' ^ ' 

Additional general conclusions may be reach^ in terms of 
the overall approach to design methods research and experimental 
technique demonstrated in 'this' study . These -are important be- 
* cause of the pilot aspects of this research in,. attempting tp 
indicate a' ne^ area f or exp^^riinental study. 4 ' 

The ability to perform design methods research Appears to 
' be strongly supported, if jnot conclusively established by this 
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study. The review of literature ^and related^ research indicating* 
thu need for this type of research 'were reinforced by the national 
suirvey of design strategies which* indicated both a level of - ^ 
CDmnion understanding about design strategies, and a signiricatnt 
variation in responses and conceptualizations which could be .r 

resolve^ "by 'objective study. In addition, the acceptance of J the ' 

■ '• • . ' . ' ^ 

test strategics by th'e subjects, and the suggestive differences 

. »* °«* 

'in performance by strategy groups indicate possibili tie's ' for 
further expeirimental studies. This stu4yf then, despite the 
problems with variability, does not diminish ^he relevance of 
design methods research. > " . > .„ 

The experimental task itself did appear to be useful in 
st)Udying the design process an'd the effect's of strategies, 
although further refinements are, indic&ted. In most cases> the 
subjects were abl^ to ' function well using the information banks 
to develop an understanding, of the test problems and develop 
meaningful spTutions. This approach does appear to generate 



a considerable body of useful data on design perfo'irmarrce which 
can be used to demonstrate .^and compare the effects of desi^gn 
-fiiethods . r 

A particular question was raise(dj.,fabout the ef £ectiv^eness ' 
of the techniques for evaluating the subjects' desi^gn* solutions.. 
The quantitative measures of solution .^characte'ris tics did not * . 
seem to adequately ^describe or drs tinguish between the different/* 
.solution configurations. «^he subje'ctive evaluations also/^did * 
not appear to be consistent in establishing 'Solution quality. 
There is appa'renbly a need for ;,thfe development gf more refined * 



, ■ ■ .-^ . ^ - , • , • . ■ 

techniques for solution evaluation both for experimental and 
practical purposes. " > - 

DISCUSSION • ■ . ' ■ 

t ' . 7-^ • . ■ ' . ' \, • . * ^ 

The major issue ra^-sed in. this study is the variability \ 

■ ■ * . « 

« * in subject perfbrmance which occurred despite extensive attempts 
. ■ - \ ' ' \ ^ / • • - ' / . ' ' ' ' 

to produce- controlled^ differentiated behavior. ' The experimenter 

• V • . • ■ 

J*' attempted to sele-c.t a highly uniform group of subjects, provide 

* ' * f?, » ' * 

• * . ' • . ° ■ ' ' • / " ■ 

a constrained set of te^t problems, utilize a standard information 

bank arid experimental task, whiTe using specifically distiirct 

» * » - * ' '■ - • ' ,'.»//' 

"strategy guidelines. Despi tfe/.this effort, the differences be- 

^ . ^- . ■ ■• . //^ ■' . ■ 

tween sub jects^ within the strategy groups were overwhelmingly 
more significant than the *dif ferences between the groups using - 
different strategics. In 'bther words/-e^en in a relatively 
limited problem^solVirig situatioa, subjects using identical 

' strategics did not^generate statistically uniform scores on/a 

' \ large percentage of measures. 

Th'e explanation for^this vari^ility is not clear at this 

<^ * " ' •> • . 

time. One explanation would suggest that the de^ign^trategies 

^ - . ^ ' . ' - * ^ ^ 

.by their nature have a negligible effect on design performance. 

This does not se^m consistent with the observed^ suggestive. 

* ' ^ ^ ' . > , i . ^ 

strategy differences. * An alternative explanation would be that 

' indi'vidua^l strategy inte3:preta.tions and ingrained personaJK design 

styles confounded the strategy effebts. This would be consistent 

with the geiierall^^ accepted concept of the design task as an 

r individual ,j intuitive , creative act. in this easo/ a given 

. strategy wo'uld likely be implemented only partially by the "^sub ject , 

Utilizing aspects-- which, were compatible .^ith the subject. 
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.personal approach, and rejecting reinterpreting conflicting 
cohorts • ' \ ^ . 

The problem :^o-r the design methodologist is- therefore to 
develop strategies which can- be acceptedTlby" individual designers 
and to present strategies in a manner whidh is pnderstandable 
and useable. The design -methods researcher 'must insure that 
the methods^^ing studied are actually implemented by the subjects^ 
and tha't indicative differences in performance are captured by 
the experimental te^chnique; ^ 
A. Design Methods: Research ^ ^ ' \ ' * A ^ * 

The results of this research should not discourage -^an / ^. , 
interest in the study of design methods, but should rather . . 
stimulate furtheis^. attempts- to develop us'eful techniques, fbr 
investigating 1^e impact of alternative methods c«i ^the design 
process;. In this regard, the results to some degree supp\.ort 
the utility of the present approach with 'further refinements, 

' . ^ \ : • 

Subject variabi-lityVould.be a major concern in further 
refinements, and could be treated in a number ^f ways. Some 
variability could be controllecJ with a mor'e extensive orientation.- 
and training program to insure that subjects could and would use 
the test methods. Patterns of variability associated with-^dif- 
ferent methods could be^ identified using larger groups- of subjects 
More sensitive performance measures could be designed to identify 
specific effects, wit:h problem formats developed to permit greater 

extremes of behavior. ^ . 

I / •■ . ' • . „ 

\ In other areas, the^time and effort limitations for 'sub jects 

could be red.ucej^ by 'designing apparatus with more convenient 



means for information presentation and manipulation , possibly 
involving^ some form of mechanicjal assist^fnce. The informati9n 
data, couia be supplemented by ai^ expanded graphic capability 
and scime fqrm of "thinking aloud" technique for recording verbal 

comments^ The information items themselves could be modified 

I ' ^. • • • 

to cover different, aspects of environmental design problem- 
solving as well as problem-solving in fields such as* Urban 
Design, Product Design, Machine Design , Inaustrial Engi;iieering. 
- With appropriate modification this ba&ic approacH could 

also be used for investigating a range of problem-solving 

* ' . * *' 

behaviors. Cpmpar,isons of^methods could be used yi^^th alternative 

strategy .configurations , as, well as different heuristics and 

decision-^making techniques. Basjic information processing could 

be studie(^ in tferms of types of information, forms of information 

presentation^ alternative options for- information. manipulation. J 

Ln addition ,\more funfiamental studies could be m^de of the 

psychological and cognitive aspects of design problem-solving. 

In the immediate future, more specific extensions, of this ' 
■ ^ ^' ^ . , . • • / ^ ^ . * ' ' 

study could also; be considered including: • : . 

1; Further analysis* of the already .collected daiia in terms of 

' the dynamics o^ information processing within the test sessions, 

identifying patterns of information useage associated with 

each strategy step. . . • 

2. Additional experimentation using a simplified experimental 
task;, with more extensive training and a larger -subject group 
to incr.^asQ the consistency of results. \ *^ * 
.3. Development and experimental comparison of refinements 



\ ' ' . ■ • ' 

in the 15-step strategy, using different strategy and problem *. 

vaisiations to determine the most effective combxnation' of steps 

*and procedures for environmental design, / ' ' ^ 

B, Design. Methods Imp-lications 

In a larger sense, this study ,and f-urther research in design 

* ^ 

methods should lead to a' better understanding and application 
of design method^- by practitioners in environmental design and . 
related f ields i[ y To^.the extent that differences between design 
strategies weref indicated, they present a . challenge to traditional 
assumptions about the negligible effects of. methods and the . 
essential similarity of alternative methods'. If real differences 
can be identified between alternative methods / tfien it is important 

to seajrch for th(a best possible- methods , testing , and comparing 

' ■ * ' ' . ' . 'i ♦ 

methods of all kinds to de.Jiermine their applicability, fot probleift- 
solving. In particular, the suggestive results of this study ^ 
appear to Support* the^pr^op'onents of more "systematic" 'methods , 
stressing detailed , operational -procedures, dir^<|ted toward a 
whole-problem viewpoint, wijth a controlled appiid|ition 'c5f problem- 
solving information. ^ s - • . ^ ' 

The pos3ibilities for Establishing meaningful differences 

' ^ .• . ' • ■■ ■ 

between design "strategies also present ^a challenge to design 

. . . i ' ■ : ' . ' :^ . /' 

educators in terms of identifyiriLg th^ most' appropriate design 
methods and presenting them as effectively as possible to design 
students/ This, suggests thp heed for ^a^ more formaii.zed design 
.methods curriculum i^nvolving a comparative analysis of alterna-- ^ 
tiva methods and their, applications to different aspects of , 
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design inf oi^mati on processing. In more general . terms this \ 

* ^ ^ • ^ . • ■ \- 

study indicates the/ importance of understanding how inform-ation \ 



is used to generate a final solution, and sug^gests teaching ^ 
approaches, using information bai;iks, and rec9rding detailed in- ^ 
v^f ormation protocols, as a means 6f demons<trating and reinforcing 
problem-solving behavior. ' ^ , ' 

The broadest implications of this research go beyond the 
cc2rn<:ern with specif ic design method? or techniques to a coilsid- 
eration of the entire design process. This .study is intended ^. 
. .^to serve as one aspect in the development of a bcmprehensive ' . 

c . ■ • . . • . ■ ' . 

^ JsciGince— Q-£^:^^ toward making the. best possible use 

of the designer's resources in producing the best possible ^ J 
'design produce. The underlying assumption of a design' science / ^ 

supported in part by Ihis research, is that design io ' a rational 

' *'.*.■ 

• ■ . i . . • ■ ' ■ 

activity which 'can be m^de explipit and- that through 3tudy and 
.understanding the ef f iciency ^nd ef f ectivjiiness ' of design activity 
ca|i be significantl^yviimproved. 
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4^0 


- 289 


• 232 


III.. 


177 


•159 


190 


■ 175 


204 


226 


113 


181 


244 


195 


400 


28C 


212 




'279 


225 


263 


256 


225 


240 


131 


199 


233 


228 


■445 


302 


252 


Sip:pificarx.ce: Problems .01; Subjects .OT; StratxProb „.01 


^.^'.2■# Cards Generated ' ' ' ' ' ' 


I 


805 


1138 


1357 


1100 


651 


593 


715 


653 


641 


416 


965 


( 674 


809 


ri. 


674 


413 


1152 


7^6 


622 


522 


440 


528 


■587 


447 


941 


,656 


64^ 


iii.^ 


397 


°376 


603 


459 


528 


'544 


605 


559 


522 


442 


746 


■ 57c 


■ 529 




''625 


642 


1037 


768 


600 


553 


587 


580 


583 


435 


884 


63^ 


.561 




: ^Problem .01; ilibaects .01; StratxProb .05 


/a Frames Required . ■ ' ' • ' 




850 


768 


846 


. .821 


787 


825 


760 


791 


947 


910 


'890 


91 6 


843 




793 


770 


303 


' 789 


'812 


759 


792 


788 


870 


901 


-865 


879 


818 


III. 


376 


440 


544 


' '453 


'7.17 


667 


740 


70a 


706 


.700 


695 


70c 


621 


ILL • 


■ 673 


659 


•731 


688 


772 


750 


"764 


762 


841 


837 


817 


.832 


761 


Sie 


•: Strategy .• 


01; P 


roblem .(*!; StratxProb .01 


'1.2.1' # Items ■ Reused " - ^ 


I. 


92 


<;-96 


206 


:''165 


81 


106 


129 


105 


' 87 


46 


16^ 


IOC 


123 


11. 


107 


5^ 


20c 


120 


92 


14 


100 


. 69 


108 


41 


127 


92 


94 


.Ill 


. 2^ 


53 


102 


60 


,99 


107 


95 


IOC 


8'2 


81 


72 


7£ 


, 79- 


j\LL 


•7^ 


101 


169 


115 


91 


■ 76 


' 108 


91 


92 


56 


122 


9C 


99 




Pi 


'oblem .1 


0; Su 


b.iect .10'. . ■ ~ . * 


. ' , =1 

1 .2^2^ # Retise' Occurrances ' „ - ■ . 


I. 


10£ 


252 


45s 


,271 


■ 133 


13'8 


341 


20^ 


169 


54 


206 


143 


206 


II. 


16 ^ 


79 


55e 


266 


152 


ie 


207 


- 126 


216 


43 


251 


^ 17c 


1.87 


t III 


. 2 -; 


8^ 


213 


. 10£ 


158 


129 


275 


181 


'125 


91 


93 


lOi 


131 


ALL 


9c 


' 13^ 


40.7 


' 21^ 


141 


95 


274 


■ 17c 


170 


63 


183 


^13S 


175 
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■". 5- 




P 






(1) (PO (3) 


(4) (5) .(6) 




(7) (B)' 


(9) 


1 


Prob. 

Total ■ 


# Groups Formed 






- 








L 

r 






128 


■241 


252 


207 


155 


.94 


,102 


1 In-/ 


121 


' 96 




' 118 




T.I.V 




75 


236 


141 


107 


68 


76 


84 


.99 


97 


160 


'119 

* * ✓ 


114 


III. 


49 


73 


153 


92 


104 


v85 


100 


96 


102 


-99 




116 


101 




■ 97 


129 


214 


147 


115 


85 


: 95 


• 97 


107. 


97 


•148 


lis 


120' 


Sig 


: Problei 


a .05; Subgect .05'; "StratxProb 


.10 








• '1.2.4 # Runs Farmed ■ ' ' ■ - 




35 


39 


'92 


■ 5,5 


55 


19 


56 


29 




'12 


?6 


• 27 




ri. 


43 


19 


39 


34 


26. 


24 


26 


26 


54 


^6 


■58 


56 


52 


III.. 


■ 16 


16 




.' 18 


24 


"56 


■ 24 


• 28 


24 


24 


2? 


25 


- 24 


' :\LT. 


« 

31 


25 


. 51 


56 


28 


26 


29 


^8 


51 


24 


54 


2S 


■ 51 
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: Pr 


otjlem .,1C; StratxProb 


.10 
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•Relatioilships 




















149 


277 


272 


' 255 


14^^ 


55' 




106 


117 


■27 


2*46 


150 


154 


EI. 


9a 


• 55 


246 


155 


95 


52 




57 


?2 


115 


170 


126 


<105' 


CII. 


145 


44 


166 


118 


• 77 


40 


1 82 


. 66 


89 


76 


145 


103 


^96 


ILL - 


131 


125 


228 


161 


105 


46 


75 


,75 


99 


75 


.186 


IIS 


118 


• Sig 


: Problem .10-, Su 


bject .05. ■ 


1.3 


.2 # Dis 


positions 




1 

\, 


f 












I. 


11^1 


234 


246 


- 198 


119 


.95 


■10^ 


• 106 


109 


?7 


- 8 




118 


II. 


122 


76 


113 


. "104 


105 


69 


'27 


67 


50 


75 


■ 57 




77^. 


Ill 


r 25 


57 


15 


52 


■71 


87 


98 


85 


42 


6j$ 


88 




61 


■ALL 


87 


122 


125 


-.111 


98 


84 


76 


'• 86 


67 


' 58 


.•51 


5"^ 


85 



Sig: Strategy .05; Problem .05; StratxProb .05 



1.3.3 # Verbal Notations \ 


I. 


5 


5 


2 


4 


5 


- 5 


7 


5 


0 


1 


• 8 




X 4 


II. 


'21 


, 17 


1 


15 


15 


-10 


4 


10 


4 


7 


,5 




9 


Ill 




10 


6 


8 


1p 


27 


16 


18 


7 


1 


6 






ALIi 


11 


11 


5 


.8 


■ 9 


14 


9 


11 


3 


3 


6 




8 
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- ■ c 














HO'O — 
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00 (5) (6) 




(7) (^0 


(9) 




V-' T. " 

Prob. 
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Grarihic 


Notations 
















I. 


1 


' 0 


0 


0 


3 


• 8 


6 


• 6 


' 0 


0 


5 


1 


2 


-.. ■ 1 — 

ri.- - 


4 


13 


" 3 


. - 7 


^ 6 


13 


3 


7 


1 


5 


2 


3 


6 


HI. 


6 


11 


' 2 


6 


it) 


19 


• 5 


. 1.1 


9 


10 


9 


9 


9 




4 


8 


2 


4 


^ -6 


13 


5 


8 


3 


5 


5 


4 


6 


Sig 


: Problem .01; Subject '.05 














Selection Groups 


I./ 


•90 


121 


100 


104 


89 


50 


18 


. 52 


,59 


74 


90 


74 


77 „ 




"46 


53 


58 


46 


49 


59 


'17 


42 


■43 


• 65 


85 


64 


50_ 


III. 


26 


39 


75 


47 


47 


46 


14 


36 


55 


50 


91 


65 


49 




54 


71 


71 


65 


62 


52 


16 


45 


52 


63 


■ 88 


68 


. 59 


Sig 


: Pr 


oblem *01; Subject .01; Stra;txProb- 


.'05 








.1.4.2 # Reuse Groups 






















38 


120 


152 


<?103 


44 


44 


84 


'57 


62 


22 


48 


44 


68 


EI. 


68 


19 


190 


92 


56 


'9 


57 


41 


49- 


26 


6? 


■ 46 


60 


rii. 


18 


32 


72 


41 


51 


36 


■ 83 


' 57 


42 




52 


"46 


48 


Ali 


• 41 


57 


1 38 


79 


',50 


30" 


75 


52 


51 


■ 31 


- 54 


45 


■ 59 


Sig; Subject .• 


P5 




















1,4 


.3 Average Size of Selection 


Groups 






> 


\ 

m 


I. 




3 


4 


■ 4 






? 


' '6 


. 4 


4 


6 


, 5 


5 


11 ; 


- 5 


•2 


5 


4 


? 


4 


8 


6 




? 


5 


4 




III 


r • 5 


• 4 


3 


4 




. 5 


' 8 


6 


■ 5 


4 


'■4 


' . 4 


5 


ALI; 


' 5 


3 


4 




4 


5 


8 


- 6 


5 


4 


? 


4 


5 


Sig: ^S 


ubn-ei,ct . 


05 ■ 






s 






• 








■ 1.4.4 Average Size of Reuse 


Groups 












I.. 


3 


2 


3 


• 3 


3 


- '5 


4 


, 3 


5 




4 


• 3 


3' 


IT. 


3 




'3 


■4 




2 


% 


5 


5 


2 


4 


. 4 


4 




. , 4 


• 3 


5 


; 3 


' 3 


3 


. 3 


3 


• 3. 


2 


■ 2 


2 


•3 


;..T,L 


. "3 


4 


$ 


3 


" 3 


3 


4 


3 


4 


°2 


4 


. ^3 


- 3 
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!• 1. . v.- l.> — J....* t^.l.J-iJT * . . 


rj'oD. f^ul) -IOC < rj • ' , jrrob. 

' (1) CP) (3) (^lO (5>' (^)' (7)<^K?^-> '(9) ^ ?otPl 


2.1,1 Problem' Descriptors Selected ^ '7; 


I. 


12 




■ 7 


7 


B 


6 


10 


. 8 




8 


'"11 


e 


8^ 




6 


7 


7 


7 




? 


12 


8 


12 


8 


V 


.1" 


9 


lit. 


• 6 




6 


5 


12 


6 


9 


9 


11 


8 


» 14 


1' 


'■• 8 




g 




. 7 


6 


10 


5 


10 


a 


• 9 


8 




•He 


8- 


Sis: Sub.iect .05 ' • ■ " 1 - ' 


III ^ 

2.1.2 Process. Des^cri" 


ptors Selected ■ " • 1 • ' 


I. 




1 

. 5 


'3 


" 8 


4 


0 


18 


7 


12 


14 


18 


\ 15 


10 


II. 


0 


2 


0 




0 


■ 0 


0 


' 0 


9 


•12 


28 


I 16 


6 , 


III. 


0 


' 1 


0 




"1 


0 


0 


0 


4 


12 


24 


13 


5 


.VLL 


6 


3 


■ 


3 


i 


0 


6 


t 

2 


^ ,8 


13 


23 




. 7 


Sig: Strategy .05; Problem .10; Subject .05 


2.1.3 Overall Reqaairements Selected . 


r. 




•15 


21 


20 


14 


' 6 


18 


13 


1I6 


26 


26 


23 


19 


ri. 


i-2 


^ 14 


16 




'20 


■ 5 


20 


1^ 


22 


16 


29 


22 


17 


III. 


"a 


7 


1^ 


9 


25 


10 


22 


19 


21 


• 21 


29 


2^ 


■ 17 




15 


12 




15 


20 


■7 


20 


16 


20 


21 


28 




18 


Sig: Subject .0 


A; St 


ratxProb_.05 « • „ 


2.1.4 General Requirements Selected . f,' 


■I.' 




15 


e 


. 12 


17 




id 


11 


■ - 

16 


16 


15 


16 


. 15 


II. 


1C 


12 


,2C 


1^ 


19 




6 


1C 


•.17 


15 


- 0 


11 


12 


Ill 




. 3 


12 


8 


14 


.5 


7 


- 9 


21 




17 


20 




ALL 


• 11 


11 


/13 


11 


17 


5 


/ 8 


10 


18 


17 


11 


15 


12 


Sig: Subject ■ ..10 ' ■ . ■ 


"2.1.5 All Definition Information Selected 


I. 




,58 


?7 


48 


4^ 


17 


56 


39 


' '49 


64 


70 


"61 


1 49 


II. 




35 


4^ 


3^ 


48 




38 




60 


51 


•70 


6C 


43 


Ill 




14 


3i 


.23 


52 


•21 


■ '38 






\62 


84 




. 42 


ALL 


4C 


29 


37 


35 


48 


17 






' 55 


"59 


75 


63 





Sig:. Strategy .01; Subject .05 





' . '1'-. — : — : ^-^v:.p ; 


, 1 . — , ■ — ir^ ' — rr-^ — r— -f — - .' ' ' 'V> ^ ' ■ 

j..>C'i:. ; .! ■\-iec '-.n ■ ■ • ir-roD. 
- (1) .(^)^^0 (-) • (7) (^0 (9)^ To'tpi 


2.?.'^ Problem Descrf- 


otors Reused . ' . 


I. 


0 


' 0 


0 


0 


5 


0 


29 


■ 11 


0 


0 


11 


. 4 


5 


II o 


2 


0 


" 0 


1 


7 


0 


11 


* 6 


5 


0 


■5 


f ' -7 


■ ' 3 


III . 


0 


• 0 


0 


'0 


6 


0 


9 


5 


. 0 


0 


7 




2 


'J I. 


1 


0 


0 


■ 1 


5 


0 


16 


7 


1 


0 


* 

' 8 




3 


Sip;: Subject K0^ - • ' , ... ^ 


2.2.2 Process Descriptors ■ Reused • ■ • 


I. 


'1 


1 


1 


.i 


0 


0 




10 


■ 5 


w 

0 


. 0 




"5 


IT. 


• 0 


0 


0 


0 


1 


0 


• 0 


0 


5 


1 


■ 16 


r 


-2 


lli . 


C 


0 

_ A 


c 


0 


0 


0 


0 


0 


0 


12 


. 0 






,111 


0 


.0 


c 


' 0 


0 


0 


10 


4 


2 


" 4 


e 


c 


■3 






2.2.3 'Overall Reqifiremeiit-s -Rei^sed --V- 


r. 


0 




^ c 


. . 5 


0 


0 


59 


20 


0 


0 


.21 




10 






6 






14 




54 


18 


:• 22 


11 


■ . 0 


11 




III. 


'2 


0 


■ 7 




12 


2 


49 


21 


4 


,4 


4 


4 


9 


lUI. 


5 


7 


v15 


8 


9 


2 


^7 


• 19 


9 


1 


. 8 




12 



2.2.^! General rvcouirements Reused 
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0 


1<1 
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0 


7. 
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4 
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.0 


■ .-,1 


c 

3 
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4 


0 


0 


■ 1 




. 0 


9 


■4 

4 


0 


0 


0 


0 
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. 0 


0 


0 


0 


■ 0 


0 


10 


3 


0 


1 


0 


0 


1 


ALL 


1 — 

, 1 




1 


2 

1 


1 


0 


9 


3 


1 


0 
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0 
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I. I 1 -26. ^1 10 ___3 . 0 126| 4.3! 9 -'0 41 ■ '1'^ ^ 23 

_I1. 14 , 6 ' 33 I '^3 _ 25 • 5 54 28 28 12 21. ■20 ' 22 

III ■ 2 0 ^ 7 3 18 2 68' 29 . ^ 17 11 

/■LL 0 11 1^^. 1o| ,15! 2I 831 331 14|, 1o| 24r i:6| 20 
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41 


64 


111 


. 1' 


1" 




■ ?';) 
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48 
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